Random Thoughts on Pooling Algorithms

When extracting information, during pooling: if one takes the regional mean (mean-pooling), it often preserves the features of the overall data and can highlight background information; if one takes the regional maximum (max-pooling), it better preserves texture features. But neither should match wavelet transforms, which can preserve more detail features and overall should be finer.

At ICLR2013, Zeiler proposed stochastic pooling: elements with larger values have a higher probability of being selected, but not like max-pooling always taking the maximum. The advantage of this method is that on one hand it maximally guarantees taking Max values, and on the other partially ensures not all elements are fooled by the Max value into causing excessive distortion.

This approach still seems to have flaws, because this random selection, though having a probability tendency, is artificially superimposed; it cannot always guarantee that among the random probability choices a better result will be selected, so worse results will also sometimes appear. But the benefit of adding a probability algorithm is that it creates the possibility of producing better results—so overall, better results are still possible.

Suppose the target is always easily hit, and there is some normal distribution approximately overlapping the target's distribution—how to ensure this distribution better matches the target? Both average and maximum will produce bias, because after all they are too brute-force; adding a probability algorithm is undoubtedly more ideal and can reduce the possibility of such bias. If luck is good enough and convergence is very good, even closer results may be obtained—so this again comes back to luck.

Yet why can convergence in the mantic arts be so accurate? From a mathematical angle this is truly hard to resolve—what exactly has not been considered? There is currently a faint line of thought: need to explore applying the mantic model—but still have not found where the bridge is. First is to find how information is calculated and folded into the hexagram; second is to find how one can still restore the information.

After trying to compute the connection between nearly a hundred thousand figures and randomly cast hexagrams, I found that establishing this connection is an extremely difficult task. Building an autoencoder that computes to a certain degree converges more and more slowly; a rough estimate would take months (and it may not even ultimately be computable). Thinking it over, the line of thought still has problems.

The road is long and winding—its distance is far.