Latest notes
Implementing an API Pool with OpenResty and Redis for Dynamic Request Allocation, Queued Waiting, and Timeout Handling
To provide a complete implementation plan, we will explain in detail how to use OpenResty and Redis to build a system that can not only allocate requests dynamically according to QPS limits, but also place excess requests into a queue for waiting when the QPS limit is exceeded, with a timeout mechanism. Complete Implementation Steps 1. Environment Preparation Make sure you have installed the following components: OpenResty: an extended version of Nginx that supports Lua scripting. Redis: used to store API QPS limits, counters, and information about queued requests.
Fixing libstdc++.so.6: GLIBCXX_3.4.30 Not Found and /lib64/libc.so.6: glibc_2.32 Not Found After Downloading llama-b3982-bin-ubuntu-x64.zip from llama.cpp
For libstdc++.so.6: version `GLIBCXX_3.4.30' not found, run sudo add-apt-repository ppa:ubuntu-toolchain-r/test; sudo apt-get update; sudo apt-get install gcc-4.9; sudo apt-get upgrade libstdc++6. For the second error, /lib64/libc.so.6: version `glibc_2.32' not found, one must build glibc from source, which is far more time-consuming—compiling llama.cpp locally is often the better option.
flash-attention
Release links for flash-attention: Linux builds at Dao-AILab/flash-attention; Windows builds at bdashore3/flash-attention.
The Easiest Way to Download and Use Llama3
1. Download Ollama. On the Ollama website (https://ollama.com/library/llama3) you can see which model tags are available. For example, to use the 4-bit quantized base version of llama8b, simply run: ollama run llama3. The finetuned quantized version is recommended for better Q&A quality: ollama run llama3:instruct. It downloads to your local machine by default, about 4.7G in size; once running, you can interact with it directly in the terminal. 2. Download Chatbox. Then in Settings, select ollama as the backend, and you can use it directly inside Chatbox.
Fixing Recurring urllib.error Messages in stable-diffusion-web-ui
When you see errors such as urllib HTTP Error 403: Forbidden, locate the ui_extensions.py file and modify the code under refresh_available_extensions as follows: def refresh_available_extensions(url, hide_tags, sort_column): global available_extensions import random import urllib.request opener = urllib.request.build_opener() # Build a list of request headers and pick one at random each time ua_list = [...] opener.addheaders = [('User-Agent', random.choice(ua_list))] urllib.request.install_opener(opener) res = urllib.request.urlopen(url) ...
Quantifying Incoherence in the QuIP Method
In the QuIP quantization method, there is a concept: the introduction of incoherence. For an existing model's parameters, requantization is the process of converting the neural network's weight matrices from a high-precision representation (such as 32-bit floating point) to a low-precision representation (such as 2-bit or 4-bit integers). This process can significantly reduce the model's storage requirements and computational complexity, thereby improving its running efficiency. However, quantization may degrade model performance, because the low-precision representation may fail to fully capture the information in the original weight matrices. QuIP addresses this problem by introducing incoherence between the weight matrix and the Hessian matrix.
Understanding an Example of LDL Decomposition
Take the original matrix: A = | 4 2 | | 2 5 |. The following steps can be used to compute the LDL decomposition. 1. Initialize a unit lower-triangular matrix L and a diagonal matrix D: L = | 1 0 | | l 1 |, D = | d1 0 | | 0 d2 |. 2. By the definition of the LDL decomposition we have A = L * D * L^T. We need suitable values of l and d that satisfy this equation. First compute the first elements of L and D: 4 = 1 * d1 * 1 +
Artificial IntelligenceThe Difference Between Layer Norm and RMS Norm
Norm is short for Normalization, an operation that normalizes data — generally into a distribution with mean 0 and variance 1. Compared to Layer Norm, RMS Norm omits one step of dividing by the mean. Layer Norm directly normalizes at the layer level, and there are generally two kinds: pre-normalization (pre norm) and post-normalization (post norm). Pre-normalization normalizes the input first and then computes the result; post-normalization computes the result first and then normalizes it uniformly.
Artificial Idiot, Eh
Chinese-Llama-2-7b-hf is extremely unreliable, runs very slowly, and often crashes. Chinese-Llama-2-7b-4bit — this one made me laugh out loud: its output just won't stop, looping endlessly. "What are husband-and-wife lung slices?" "Husband-and-wife lung slices refer to lung slices between a couple, that is, the lung structure between a couple. Husband-and-wife lung slices refer to the lung structure between a couple, which is the lung structure between a couple........." chatGLM2-6B is also pretty good.
Steps for Using DeepSpeed on Windows 11
git clone https://github.com/microsoft/DeepSpeed.git — then you need to modify some code: 1. Open the file csrc/quantization/pt_binding.cpp in the directory, and change std::vector<long int> on line 203 to std::vector<long long>. 2. Open the file csrc/transformer/inference/csrc/pt_binding.cpp in the directory, and on line 534 ...
A Small Idea Sparked by ChatGPT's Text Summarization Ability
Introducing the various races of D&D in a humorous and blunt manner: Dwarves: A mix of blacksmiths and drunkards, for whom the meaning of life is smithing, drinking, and arguing over which foam is better: beer foam or bath foam. Orcs: A bunch of big bruisers whose insight is usually "if you don't understand it, hit it hard." They once tried to open a restaurant, but because the cooking steps for every dish were "smash, fry, boil, roast," they eventually switched careers as wall-demolition specialists. Gnomes: They invent all kinds of bizarre machines, but strange malfunctions often occur. Once, an automatic onion-cutting machine suddenly turned into a time machine and sent the whole clan back to the Jurassic period.
wget Parameters for Skipping SSL Certificate Checks
Using only wget --no-check-certificate is not enough. You need to use: wget --secure-protocol=TLSv1_2 --no-check-certificate {url}
Careers That May Thrive in the Age of Artificial Intelligence
Science and information technology industry — AI model evaluator: responsible for evaluating and testing models generated by artificial intelligence, detecting their performance and effectiveness, and providing suggestions for model optimization and improvement. AI model supervisor: responsible for supervising and managing AI-generated models, ensuring they follow ethical, legal, and social norms during application, and preventing misconduct and bias. Dataset cleaner: responsible for cleaning and processing datasets used to train AI-generated models, ensuring data quality and accuracy, and optimizing training effectiveness.
What Is the Difference Between LORA and P-Tune, Two Fine-Tuning Methods?
LORA and P-Tune are both methods for fine-tuning machine learning models, but their specific implementations differ. LORA (Layer-wise Relevance Propagation) is a fine-tuning method based on hierarchical relevance propagation. In LORA, importance weights between each output feature and input feature are first computed, and the model is then fine-tuned through these weights. This approach can help deep neural network models better understand the features of input data and improve accuracy. P-Tune…
An Idea for a Loop Neural Network
A loop neural network is a special kind of recurrent neural network composed of multiple loops of different lengths. Each loop is a recurrent unit that can be seen as a simplified LSTM or GRU. Each loop can capture a periodic change at a specific frequency in the data, while the combination of multiple loops can represent more complex temporal patterns. For example, if we want to predict stock prices, we could design a loop neural network with three loops — one of length 5 representing weekly periodic change, another of length 20 representing monthly change, and the last of length 60 representing quarterly change.
A Quick Test of Content on Plum Blossom Numerology Generated by a Newly Trained AI Model
Plum blossom numerology (Meihua Yishu) is a simple and easy-to-learn divination method that can be used to answer various questions in life, such as work, relationships, and health. Its principle is based on the variation patterns of the sixty-four hexagrams — by randomly drawing plum blossom cards or rolling dice, one obtains a hexagram image, and then analyzes and judges according to the hexagram's meaning and the real situation. In this article I will present a Meihua Yishu case in which someone asks about work and obtains Fire over Heaven: Great Possession changing to Earth over Heaven: Peace.
Modeling the Five Elements' Relationships and Reverse-Solving with Backpropagation
import numpy as np # Generation and control relationship matrix matrix = np.array([ [0, 1, 0, 0, -1], [-1, 0, 1, 0, 0], [0, -1, 0, 1, 0], [0, 0, -1, 0, 1], [1, 0, 0, -1, 0] ]) # Interaction strength between each element weights = np.random.randn(5, 5) # Define the model function def
Chinese-Model Alternatives to ChatGPT: A Survey of the Latest Models That Actually Run and Train on a Personal Computer
BELLE-LLAMA-7B-2M Long before most people thought of it, this author began experimenting with Chinese training — translating Stanford's dataset into Chinese and using it as Chinese corpus for further training LLAMA. The earliest Chinese-trained version was therefore one this author trained with PEFT using LoRA and uploaded to Hugging Face, as the model alpaca_7b_zh. This model has one drawback: it runs rather slowly. With GPU inference it will crunch away for ages before finally producing a single result, roughly
A Look at Several AI Code-Completion Plugins for Visual Studio Code
1. CodeGen in the PaddlePaddle suite claims its 16B model beats OpenAI's code-generation model. By the model name it comes from Salesforce, and the models are also on Hugging Face — search CodeGen there for models of various sizes. For use in China, going straight through PaddlePaddle is simpler since downloads are fast. You can use the FauxPilot plugin — good and free — though you need to configure it yourself; it isn't complex. 2. Amazon CodeWhisperer
Domestic Mirrors for PyTorch
Example: https://mirror.sjtu.edu.cn/pytorch-wheels/cu113/?mirror_intel_list
Quickly Install the transformers Source Code via Tsinghua's pip Mirror
Because of network issues, downloading the source and installing directly is extremely painful, so you can first install via a mirror. For example: pip install transformers -i https://pypi.tuna.tsinghua.edu.cn/simple This installs all the dependencies. Then pip uninstall transformers, then git clone https://github.com/huggingface/transfor
Using bitsandbytes on Windows 11
To load large models in quantized form you need the bitsandbytes library, but it was originally designed to be installed and used under Linux. So to use it on Windows 11 you need to make a few changes. After pip install bitsandbytes, modify the source in two places. With conda installed, the directory is usually: {driver}:\conda\envs\{name}\Lib\site-packages\bitsandbytes\. After pip install, the output usually says where it was installed — just copy that path.
Artificial IntelligenceTrying Out the 7B and 13B Models of LLaMA
LLaMA has four pretrained model sizes: 7B, 13B, 33B, and 65B. The 33B and 65B require far too much VRAM to try, so I could only test the 7B and 13B on a 3090. Limited by VRAM, I used the INT8 loading approach, mainly via llama-int8: it loads the model into system memory first, then copies it into VRAM. The 13B model used about 17 GB of VRAM, and the running speed was not ideal; testing its text generation was also disappointing — nothing stood out.
Data AnalysisSome Simple Thoughts on Neural Networks
Put simply, a model is really just a converter — or rather, it plays the role of a complex function that transforms input data into another form of presentation. If the transformed form is intuitive enough for the corresponding task, that task becomes easy to accomplish. Looking at things now, meta-learning has diverged somewhat from the original vision: early meta-learning aimed to let machines learn how to learn, but that was only a concept — in practice it has not truly been achieved. In ordinary transfer learning, one usually trains a model for a task first, then
If We Represent the Generation and Overcoming of the Five Elements with an Adjacency Matrix
The Five Elements' generation cycle, as a directed cyclic graph, can be written as an adjacency matrix. So can the overcoming cycle. Clearly, in matrix form, shifting the generation matrix one step right yields the overcoming matrix, and shifting the overcoming matrix one step right yields generation again. The one-step-right shift matrix is given below.
Some Unexpected Revelations of ChatGPT
Human experience often blinds us to things right in front of us, but using AI can often solve that problem. For example, today I suddenly thought to ask ChatGPT an interesting question: how does modular arithmetic convert into polar form? Its answer explained the difference, then erred in a later modular cycle calculation—apparently defaulting to an extra +1. Here is a transcription of the exchange.
A Free Alternative to GitHub Copilot
For Visual Studio Code, the usual choice is the FauxPilot plugin with a FauxPilot server—but after installation the FauxPilot plugin is not especially usable. A better approach is to run FauxPilot yourself, install the GitLab plugin, and point its code-assist URL at your server. To avoid a painful environment setup, Baidu PaddleNLP can start the code-assist service in one line; models of 350M, 6B, and 16B parameters are supported.
Configuring the Tsinghua Mirror for Conda
Add the TUNA Tsinghua Anaconda channels with conda config, set show_channel_urls to yes, or edit .condarc by hand with the default and custom channel URLs below.
AI Voice Changer
Recast - Koe Recast is an AI style-transfer converter for speech. A pity its Chinese support is not ideal: the converted style feels like a foreigner in a film speaking somewhat poorly enunciated Chinese. Both male and female voices were tested, neither fully satisfying. For English speech conversion, however, it is at its peak—arguably the voice changer the public actually wants.
A Universal Fix for Python Installation Failures or Extremely Slow Downloads via pip
Visit https://pypi.org/simple/<package-name> directly, download the .whl file, and install it. For example, to install onnx, go straight to https://pypi.org/simple/onnx — apply the same pattern for other packages.
Pre-trained Models Required by GFPGAN
RealESRGAN_x2plus.pth, GFPGANV1.3 under experiments\pretrained_models\, detection_Resnet50_Final.pth, parsing_parsenet.pth under gfpgan\weights. Related commands: Inference — python inference_gfpgan.py -i inputs/whole_imgs -o results -v 1
Artificial IntelligenceBaidu Yige Text-to-Image Generation
Baidu has another text-to-image API called Yige (一格), which is relatively more free — at least it can generate surrealist-style images. Generation takes longer here, but the surrealist images it produces are quite decent.
Artificial IntelligenceText-to-Image Generation Results with Baidu Wenxin
It's clear that the generation still has quite a few issues: the character's head remains less than ideal. The background is still quite good and the clothing is fine, but the face falls short. What especially needs noting is that after switching to oil-painting style, the results — especially faces — remain hard to control; this is a common flaw of diffusion models. Sometimes it does produce acceptable images, and overall, cartoon-style faces are relatively acceptable. So in the keywords, I tried covering the face, but unfortunately Wenxin doesn't seem to recognize such instructions — though unexpectedly, the results were still pretty decent.
Data ForecastingExploring Why Ordinary Generative Adversarial Networks (GANs) Struggle to Simulate the Operations of Metaphysical Divination
In a previous post, someone asked in a comment whether generative adversarial networks could be used to simulate she fu (a divination method of guessing hidden objects). I experimented with this briefly before, and the results were less than ideal. There are reasons for this: the implementation of an ordinary GAN is essentially a transformation or mapping from a latent space to a sample space, and when learning high-dimensional data it often exhibits certain deficiencies — intuitively, it usually cannot well simulate the concepts of cycles and periodicity. For example, suppose we try to learn such a clear distribution:
Data ForecastingHow to Teach Computers to Learn Zhou Yi Image-Association — Plus Some Thoughts on AI Research
For many people, teaching a computer to perform quxiang (image-association) sounds mystical, but it is not difficult at all once you understand the principle. From an abstract perspective, the classification of images in Yi studies is simply a way of categorizing all things. Because the sages observed the opening and closing of yin and yang to name things, when humans name the objects they perceive in the objective world, they always follow some intuitive regularity. And this regularity transcends language — in recent studies of language translation models, one can find similar interesting phenomena: a model for translation between two languages can be transferred to another pair of languages, adjusted, and then put to use.
Batch Script FFMPEG Video Cutting
I originally followed a method found online, but after cutting the videos, they would not play on H5. Further investigation revealed that each browser has requirements for which H.264 encodings it supports, and iOS Safari in particular is stricter—so I record it here as follows: setlocal enabledelayedexpansion; set x=0; for %%i in (.azhi\*) ^ do ( set /a x+=1 set new_dir=G:\course\bazhi\!x! mkdir !n...
Reflections on Using SVMs with Small Samples
Today when validating test data, I found that when using cross_val_score for cross-validation, the larger the proportion of the randomly allocated test set, the higher the accuracy—and the smaller the test set, the more likely the average accuracy is to be low. There are two major pitfalls when deriving models with SVM. One is that after train_test_split, when the data volume is insufficient, the learned sample distributions differ greatly and are extremely unstable; how to find the optimal train_test_split is hard to determine. The other is whether the assessment of the model’s generalization ability is reliable—and that is bound up with the dataset partitioned by train_test_split. This directly means that occasionally scoring high on the test set is merely an illusion...
Key Points When Calling WinRT from .NET Core
These past two days I wanted to write a simple video-compositing tool. If it weren't for distributing it to others, I could have finished it directly with an ffmpeg script or Python. But now this need has arisen, so I considered finishing it quickly with .NET + WinForms. Video editing in WinRT looked quite simple, so I planned to write a small tool with WinUI 3 — only to discover this thing has murky ties to UMP. For personal use, deployment is super cumbersome; the easiest way is to publish to the Microsoft Store and download from there. What kind of joke is that?
Fixing "Kernel Died with Exit Code 1" for Jupyter Notebook in VS Code
When you see the message Kernel died with exit code 1, the error usually looks something like this: traitlets.py:2195: FutureWarning: Supporting extra quotes around Unicode is deprecated in traitlets 5.0. Use 'hmac-sha256' instead of '"hmac-sha256"' – or use CUnicode. warn(C:\Users\rohit\AppData\Roaming\Python\Python38\site-packages\traitlets\traitlets.py:2150: FutureWarning: Supporting extra quotes around Bytes is deprecated in traitlets 5.0.
Programming NotesSolving Unclear Plotting in RStudio on Windows 10
First install R with: sudo apt-get install r-base. After installation, if plots under RStudio on Windows 10 are rather unclear—circles and lines with jagged edges—switch to WSL2 with Ubuntu as the backend and use RStudio Server instead.
Ten Key Points of Causal Diagrams
When modeling events, whenever possible, try to build a directed acyclic graph (DAG). 1) Each node on a DAG corresponds to a random variable, not its realized value. 2) The presence or absence of arrows in a DAG corresponds to the presence or absence of individual causal relations in the population. 3) "Non-manipulable" variables and their arrows should be drawn with care. 4) It is best to draw a DAG for the total population rather than for exposed or unexposed groups, so that bias can be better assessed. 5) DAGs mainly help study whether confounding exists in the assignment of interest, with respect to the concept of confounding. 6)
Programming NotesLocal Area of a Pentagram
[latex] \frac{10 \tan\left(\frac{\pi}{10}\right)}{3-\tan^2\left(\frac{\pi}{10}\right)} r^2 [/latex] Other properties: The interior angles of the inner hexagon are (n-2)*π)/n; each marked angle is: every marked angle equals 3π/5.
Optimal Speed Settings for Swap on Linux
To find the best block size for creating a swap file, use a simple dd benchmark that measures transfer rates across block sizes from 512 bytes to 64MB. In repeated tests, 128K was generally the best size; 100000 blocks of 128K yields about 12GB. Create virtual memory with: dd if=/dev/zero of=/tmp/swapfile bs=131072 count=100000; mkswap -f /tmp/swapfile; swapon /tmp/swapfile.
Fun: Letting GPT-2, an AI, Read the Daoist Scriptures
I tried OpenAI's GPT-3 engine. What makes GPT-3 distinctive is its scale of operation and its ability to complete an incredible range of tasks; after trying it many people exclaimed that writers may soon be out of work. I also tried the Qingyuan CPM Chinese model based on GPT-2, and was surprised by the results—shocking, in fact. For example, given the prompt about the muddy path and the little stone bridge of one's hometown, and letting the AI continue, it wrote content like this.
After npm run build, Serve as HTTP with Alpine and nginx in Docker
To implement a simple CI setup that automatically builds and deploys to a target server after code commits, after deploying Gitea and Jenkins you still need further steps. With Node.js, after npm run build you get pure HTML in the dist directory; using npm install http-server -g yields a Docker image up to 100M. Switch to Alpine with nginx for deployment—size drops to about 13M.
Downloading PyTorch with Conda and Installing Offline
How to download PyTorch via conda using Tsinghua mirrors and install it offline: (1) Install Anaconda. (2) Add Tsinghua mirrors. (3) Start an online install, then interrupt after choosing y. (4) Check urls.txt under Anaconda3\pkgs, download packages, and install locally with conda install --use-local. Finally in the activated environment run conda install -c local pytorch cudatoolkit=10.2.
Flutter Configuration
Flutter looked quite interesting, so I downloaded it to try. After installing Android Studio and the Flutter plugin in Visual Studio Code, pressing F5 as instructed did not work; trying a real device also failed. These steps are needed to get it running in one go: (1) Download Android Studio. (2) Download Flutter. (3) Add environment variables. (4) Download and configure Gradle, preferably set it directly in Settings so the wrapper download is not too slow. (5) Modify the project's Gradle: comment out google() and jcenter() in build.gradle and add Aliyun Maven mirrors; match the classpath version to your Android Studio. (6) In the SDK, change MAVEN_REPO in flutter.gradle and resolve_dependencies.gradle to http://download.flutter.io.
Quantum Computing and the Four Images (Part Three)
Setting aside all the mathematical formulas and examining qubits purely in themselves, things become more direct. One qubit within a pair essentially has six states: |0⟩, |1⟩, |+⟩, |−⟩, |i⟩, |−i⟩, and these six states can be intuitively represented with the Bloch circle. But when examining things at the level of a qubit pair, the combinations of quantum states amount to thirty-six. First, |0⟩ = [1,0], |1⟩ = [0,1], which are the |0⟩ and |1⟩ states — the yin and yang states mentioned earlier. When in |0⟩ and |0⟩…
Programming NotesQuantum Computing and the Four Images (Part Two)
If quantum operations are regarded as something akin to circuits, since a single qubit has not only a yin state and a yang state but also mixed states, controllability can be achieved by exploiting the interactions among them. So one can design gates for logical control, and the most fundamental cases requiring operation are four: unchanged, flipped, equal to 0, and equal to 1. Put simply, an operation is essentially controlling a quantum state to transform into another quantum state — this process can be called evolution, and evolution is in fact a rotation in Hilbert space. Rotations in Hilbert space can be performed with matrices, and the matrices used are unitary matrices — also called unitary transformations — so here they can be called U operations…
Quantum Computing and the Four Images (Part One)
An ordinary bit has only two states, 0 and 1, whereas a qubit can have more states — but it also has the two cases of 0 and 1, usually recorded as |1⟩ or |0⟩. If expressed with matrices, they are respectively…
101 posts