Latest notes
Data ForecastingExploring Why Ordinary Generative Adversarial Networks (GANs) Struggle to Simulate the Operations of Metaphysical Divination
In a previous post, someone asked in a comment whether generative adversarial networks could be used to simulate she fu (a divination method of guessing hidden objects). I experimented with this briefly before, and the results were less than ideal. There are reasons for this: the implementation of an ordinary GAN is essentially a transformation or mapping from a latent space to a sample space, and when learning high-dimensional data it often exhibits certain deficiencies — intuitively, it usually cannot well simulate the concepts of cycles and periodicity. For example, suppose we try to learn such a clear distribution:
Data ForecastingHow to Teach Computers to Learn Zhou Yi Image-Association — Plus Some Thoughts on AI Research
For many people, teaching a computer to perform quxiang (image-association) sounds mystical, but it is not difficult at all once you understand the principle. From an abstract perspective, the classification of images in Yi studies is simply a way of categorizing all things. Because the sages observed the opening and closing of yin and yang to name things, when humans name the objects they perceive in the objective world, they always follow some intuitive regularity. And this regularity transcends language — in recent studies of language translation models, one can find similar interesting phenomena: a model for translation between two languages can be transferred to another pair of languages, adjusted, and then put to use.
Fun: Letting GPT-2, an AI, Read the Daoist Scriptures
I tried OpenAI's GPT-3 engine. What makes GPT-3 distinctive is its scale of operation and its ability to complete an incredible range of tasks; after trying it many people exclaimed that writers may soon be out of work. I also tried the Qingyuan CPM Chinese model based on GPT-2, and was surprised by the results—shocking, in fact. For example, given the prompt about the muddy path and the little stone bridge of one's hometown, and letting the AI continue, it wrote content like this.
Data ForecastingResidual Networks
I came across an explanation on Zhihu saying that the basic form is actually similar to differential amplification, which makes a lot of sense. Suppose F'(5)=5.1 — the rate of change from 5 to 5.1 is extremely low because the base 5 is too large, so it can be split: H(5)=5.1, H(5)=5+F(5), F(5)=0.1. At this point, because 0 becomes 0.1 in the split term, the rate of change reaches 10%, amplifying this effect. Since neural networks cannot well identify such a split, artificial structure design is needed to assist.
Chatting about the Zhouyi Cantong Qi (10): The Jianmu Connects Heaven and Earth; the Wondrous Way Comes from the Spiritual Root
Internally nourish the self — quiet, still, empty and void. Return to the root, hide the brightness; inwardly illuminate the form-body. Seal the openings; fortify the spiritual plant. The three lights sink on land, warming the child-pearl. Look at it and it is not seen; near, it is easy to seek. Within the yellow, principle gradually opens; moisture reaches the skin and flesh. A correct beginning leads to a correct end; with the trunk established, the tip can be held. It is concealed by the One; the people of the world do not know it.
Data ForecastingStudying CNTK (6): ResNet
To unleash the power of image recognition you still need the ResNet architecture, first proposed by Microsoft Research Asia. This network structure can achieve recognition rates above 90%—for example, ResNet CIFAR-10 needs as many as 21 convolutional layers, with batch renormalization and normalization redone at every step. In August 2016 Google open-sourced Inception-ResNet-v2, based on TensorFlow, with even stronger recognition that can accurately classify Alaska Malamutes (left) and Siberian Huskies; there are also V3 and V4, with deeper architectures. Notably, Inception-v4 has no residual connections but matches V2's performance.
Studying CNTK (5): ConvNet_CIFAR10.cntk and ConvNet_CIFAR10_DataAug.cntk
In terms of tasks, object recognition is the hardest, because object size and appearance change flexibly. Traditional neural networks find this almost impossible, while convolutional technology overcomes this—though it still depends on massive compute. The model in ConvNet is: model = Sequential ( Normalize {featMean, featScale} : ConvolutionalLayer {64, (3:3), pad = true} : ReLU : ...
Data ForecastingStudying CNTK (Part IV): ConvNet_MNIST.cntk
The results show: Minibatch[1-10]: errs = 0.470% * 10000; ce = 0.01797164 * 10000 Final Results: Minibatch[1-10]: errs = 0.470% * 10000; ce = 0.01797164 * 10000; perplexity = 1.01813410. The error rate is only 0.47%, which means an accuracy of 99.53%.
Data ForecastingStudying CNTK (Part III): MNIST Recognition with 02, 03 and 04
The only difference from a neural network with a single hidden layer shows up in the MODEL definition. In 01 it is defined as: model(x) = { s1 = x * featScale; h1 = DenseLayer {200, activation=ReLU} (s1); z = LinearLayer {labelDim} (h1) }. In 02 the definition makes more sense alongside the diagram.
Studying CNTK (3): MNIST Recognition with 01_OneHidden.cntk
# Parameters can be overwritten on the command line # for example: cntk configFile=myConfigFile RootDir=../.. # For running from Visual Studio add # currentDirectory=$(SolutionDir)/<path to corresponding data folder>
CNTK Study (Part II): Testing Several MNIST Recognition Results
01_OneHidden uses a single hidden layer, equivalent to a simple three-layer neural network, with ReLU applied directly — the result is quite good. Minibatch[1-10]: errs = 1.560% * 10000; ce = 0.05553107 * 10000 Final Results: Minibatch[1-10]: errs = 1.560% * 10000; ce = 0.05553107 * 10000; perplexity = 1.05710186
CNTK Study (1): Converting MNIST Files
In the CNTK MNIST example, there is a .py file that first converts the image files into a text representation so CNTK can read them. File location: \cntk\Examples\Image\DataSets\MNIST\mnist_utils.py. import sys import urllib import gzip import shutil import os import struct import numpy as np ...
Handling Floating-Point Precision Problems
When converting to a JD value during calculation — for example, for 00:00:00 on January 1, 2000, the JD value is 2451545.4166707597 — restoration yields December 31, 1999, 23:59:59.999986588954926, due to floating-point arithmetic issues. After a manual correction adding one second (1/86400), the seconds become 0.999981164932251111; in other words, the stable floating-point value here is around 0.99998. Since 1000 milliseconds make one second and astronomical calculations cannot be that precise anyway, the computation can be simplified.
Obtaining True Random Numbers with Intel CPUs
Intel built into its third-generation Ivy Bridge Core processors a feature that obtains hardware true random numbers using resistor thermal noise. The principle of the code here is to read a 16-bit random number directly into the rax register via assembly instruction. In 2013, Kyle Condon of the UK launched a petition on Change.org asking maintainer Linus Torvalds to remove RdRand from /dev/random in order to improve kernel security. Because RdRand is an instruction used to generate random numbers, included in the Intel 64 and IA-32 instruction set architectures, and the cryptographic standard it relies on—NIST SP800-90 set by the NSA—is suspected of containing a backdoor. Linus Torvalds responded to the petition swiftly, blasting the petitioners and calling them far too ignorant. Because in Linux systems the random number generator at /dev/random applies further randomization on top of RdRand's random numbers, which not only avoids the possibility of a backdoor but also improves the quality of the generated random numbers. There are also ready-made JAVA libraries online: https://github.com/cambecc/drnglib
USC Researchers Say Artificial Intelligence More Accurately Predicts Whether Couples Are About to Divorce
Reposted from NetEase. According to foreign media reports, artificial intelligence can more accurately predict whether your marriage is in crisis than a therapist can. A new type of computer algorithm can predict whether a marital relationship is developing cracks based on the tone of voice when people speak. This algorithm predicts marital success with accuracy as high as 79%, more accurate than data provided by many therapists. The algorithm was developed by researchers at the University of Southern California, who studied hundreds of dialogue exchanges from 134 couples in marriage counseling and tracked these couples’ marital conditions over five years, thereby drawing their conclusions. The algorithm breaks with traditional dialogue patterns; through speech processing technology it records acoustic features such as voice frequency, pitch, intensity, and jitter. Based on this data it measures whether a person has emotional problems. These features will be used in couples psychotherapy to prevent divorce.
Data ForecastingRevelations of the Yin–Yang Law in the Cellular Automata of the Game of Life
These past couple of days I have been trying to play with the Game of Life. The game itself is very simple—just iteration of some simple rules—but under certain parameter conditions it can produce complex emergence, which is a truly remarkable point. Generally speaking, mechanical calculations often yield mechanical results, but under specially constructed parameters subtle phenomena can arise and complex behavioral rules can emerge. Its greatest significance lies in the fact that these complex and hard-to-predict rules are produced by iterating extremely simple rules. This illustrates a philosophical point: not all complex phenomena arise from complex causes, but neither do all simple rules necessarily iterate into complex results—only when things fall subtly and precisely under certain conditions can such effects be produced.
About Coordinates in Swiss Ephemeris
2.4.4. Coordinate systems, degrees and radians SEFLG_EQUATORIAL returns equatorial positions: rectascension and declination. SEFLG_XYZ returns x, y, z coordinates instead of longitude, latitude, and d
NIST样本库的识别率达到了100%
现在只怀疑，是自己的代码写错了......可能是有点兴奋，反复检查，怎么都觉得代码没错，可是原理不对劲啊.... 从经验来说，一定是哪里搞错了，决定休息了，明天再检查代码。
Data ForecastingSolved the Problem of Using True Random Numbers for Divination
For divination systems based on spiritual intuition, to achieve accurate predictions, one must first find data that is currently unpredictable, and then derive predictable results from this data. This is a rather contradictory logic. The unpredictable data is naturally true random numbers. As for whether true random numbers actually exist or not, it is difficult to say; it is a philosophical issue. Although everything is a manifestation of probability, what determines probability? Back to the topic, initially I checked Taobao for hardware true random number generators, but the omnipotent Taobao was disappointing. Then I checked the computer's CPU and found it was an Intel CPU,
Data ForecastingImplementing XOR Logic Using the BP Neural Network Library in AForge.NET
When debugging the neural network, it felt like no matter how I tested the data, convergence was extremely slow. I kept suspecting there might be an issue with AForge.Net's BP algorithm, so I tested the XOR logic. The code is as follows: private static void testXOR (){ var func = new AForge . Neuro . SigmoidFunction () ; var network = new AForge . Neuro . ActivationNe
Data ForecastingSingle-Layer Perceptron and Simple Thinking in Philosophy
I accidentally came across the single-layer perceptron again. To deepen my understanding, I wrote code to test it.
A Quick Look at MNIST Training Accuracy Statistics
6-layer NN 784-2500-2000-1500-1000-500-10 (on GPU) [elastic distortions] none 0.35 Ciresan et al. Neural Computation 10, 2010 and arXiv 1003.0358, 2010 committee of 25 NN 784-800-10 [elastic distortio
Data ForecastingHandwriting recognition is still not very good
在Accord.Net中，尝试用SVM向量机跟深度网络两种去识别手写字体，效果都不是很理想，而按目前所知，效果最好的恐怕还是卷积神经网络了，卷积神经网络最近看了一下原理，大体明白了，不过自己手工实现时，遇到了不少问题，还需要进一步研究。
Data ForecastingDeep Neural Network Learning: Linear Regression and Logistic Regression
Although I roughly understood the method of Sparse Coding, I always felt there were some inconsistencies. According to my master's teaching back then, whenever I encountered a problem that I couldn't figure out, it was always because my foundation was not solid. So, I reviewed linear regression and logistic regression to clarify my understanding. Linear regression, in concept, is actually quite simple. It can be abstracted into a geometric concept. For example, on a plane with a bunch of points, if you can connect all of them with a curve, then this is a linear problem because the curve can cover them all, and linear problems can be solved through linear regression. As for the term "regression"
Data ForecastingDeep Neural Network Learning: AutoEncoder Sparse Coding
This is a technology that feels almost magical. Given a set of images, software can scan them and extract features. For humans, this is easy, but for software, the biggest question is: how should it be done? The technology behind writing software isn't particularly high-tech; ultimately, it relies on human thought. We convert these thoughts into code to solve real-world problems. Programming can be said to be an excellent way to verify ideas. I even believe that Daoists of the new era should each master programming skills. I've gone off-topic. This gives a pile of images, and then can extract features
Data ForecastingAnalyzing the Principle of BP Backpropagation Algorithm Based on the Source Code of AForge.NET from Google
In the AForge.NET neural network model, the BP learning algorithm directly uses the Activation Network class instead of an interface. As a result, using the BP algorithm requires inheriting from the Activation Network class to override function calls. However, inheriting from this class causes many complications, making it inflexible to modify the network structure or adjust the algorithm. In this Google library, the neural network algorithm is relatively simple. Its general structure consists of multiple stacked layers, with neurons contained within these layers. You create an Activation Network object and then specify how many layers to divide it into.
Data Forecasting道家卜卦思想与稀疏表示的联系
在<< 分享在马尔代夫的修行感悟 >>一文中谈及"仔细观察海洋，在高高的天空，一样能看到海洋的波浪，这就是自然界的特点，离远了可以看到事情的整体状态，而离近了，则可以看到事物的细节状态。在同一个观察高度下，是不能既把握细节，又能把握整体的，这说明一个道理，平时在思考事物时，不但应该从高的角度去看其整体，也应该从放下足够低的姿态去了解细节，这样才能得到对事物更完整的把握。用卦来观察事物时，也是这样的
World Earthquake Data Statistics Table
Statistics from 1999 to May 1, 2015 Date Time Latitude Longitude Depth Magnitude Reference Location (deg.deg) (deg.deg) (km) 2015-04-30 18:45:06.8 -5.4 151.8 60 M6.7 New Britain Region 2015-04-29 00:39:38.7 -20.9 -178.6 580 M6.2 Fiji Islands Region 2015-04-27 20:35:54.9 27.0 88.3 7.9 M4.9 India
28 posts