Chinese-Model Alternatives to ChatGPT: A Survey of the Latest Models That Actually Run and Train on a Personal Computer

BELLE-LLAMA-7B-2M

Long before most people thought of it, I had the idea of training in Chinese, so I translated Stanford's dataset into Chinese and used it as Chinese corpus for further training of LLAMA.

The earliest version trained with Chinese was therefore one I trained with PEFT using LoRA and uploaded to Hugging Face; the model is alpaca_7b_zh.

This model has one drawback: it runs rather slowly. With GPU inference it will crunch away for ages before finally producing a single result — probably because only a few million parameters were tweaked in an attempt to align it with Chinese. Given that the model was already weak at Chinese in the first place, there is little choice but to endure the output.

The greatest flaw of the LLAMA model is that it has never undergone systematic training on Chinese data at all. So even after further finetuning on a Chinese dataset, only those already-trained fixed tasks perform well; on random questions outside the training set, its Chinese is a complete mess.

Some people claim that INT4-quantized LLAMA-30B performs quite well, but with 30B parameters it is impossible to further train on a home computer, so that option loses its meaning.

If you want a Chinese LLAMA model good enough to use, you would have to retrain the whole thing on high-quality Chinese corpora — unrealistic for an individual, so this can simply be set aside.

Naturally many people noticed these issues, so some turned to other models and used Stanford's dataset for further training.

At the same time, some wondered whether training an expanded Chinese dataset on the original LLAMA could improve its Chinese ability — and so BELLE-LLAMA-7B-2M came about.

This model had only just been trained a few days prior. Regrettably, it too is not good at free-form Chinese Q&A.

Most importantly, in terms of licensing, derivative works of LLAMA may not be released directly; they can only be distributed in indirect form via a patching mechanism.

BELLE_7B_2M

This one was trained with BLOOM-7B as the base model, using the expanded Stanford dataset translated into Chinese.

BLOOM is a multilingual NLP model developed by the BigScience organization of 1,000 scientists worldwide. The large model originally trained had 176B parameters; for ease of use, some smaller variants were also released.

In simple tests, BELLE-7B-2M's Chinese performance is acceptable, for example:

Please generate a dialogue between a real-estate agent and a client.

Agent: Hello! I'm an agent with XX Real Estate. What would you like to know?
Client: I'm looking for an apartment where I can live comfortably and safely.
Agent: We have many such listings. I can introduce a few options that suit you.
Client: Sure, tell me some details.
Agent: This apartment is in the city center, within walking distance of many shops and restaurants, and transportation is very convenient. It is also equipped with modern amenities and a security system to make your stay safer and more comfortable.
Client: Sounds great. I'd like to see photos and videos.
Agent: Of course — here are several listings in the area; you can pick the apartment that suits you best.
Client: OK, I'll think about it.
Agent: Thank you very much for your inquiry. If you have any other questions or need help, feel free to contact me anytime.

In real testing, however, when generating code it produces a mix of Chinese and English — a problem caused by flawed training technique. If you truly want to use it for that, you probably still need to retrain it yourself.

Fortunately, BLOOM itself has decent Chinese support, so with its 7B parameters a home computer can still train it via PEFT with LoRA — arguably an ideal solution.

As a derivative of BLOOM, its use must follow the relevant terms: roughly speaking, you may use it as long as you do not break the law, harm others, or create negative social impact.

In fact, all models carry this requirement by default, so the BLOOM family is comparatively permissive.

Tsinghua's Chinese model Chat-GLM-6B

In actual use, Tsinghua's conversational model is among the better-finetuned ones currently available, and a low-VRAM version is provided as well. Its generated example looks roughly like this:

Not being able to sleep at night can make you feel anxious or uncomfortable, but here are some methods that can help you fall asleep:

1. Keep a regular sleep schedule: A consistent schedule helps you build healthy sleep habits and makes it easier to fall asleep. Try to go to bed at the same time each day and wake up at the same time.
2. Create a comfortable sleep environment: Make sure the environment is comfortable, quiet, dark, and at a suitable temperature. Use comfortable bedding and keep the room ventilated.
3. Relax body and mind: Do relaxing activities before bed, such as taking a warm bath, listening to gentle music, or reading interesting books. This eases tension and anxiety and makes sleep easier.
4. Avoid caffeine: Caffeine is a stimulant that harms sleep quality. Avoid drinks containing caffeine — such as coffee, tea, and cola — before bed.
5. Avoid non-sleep activities in bed: Doing things unrelated to sleep in bed — watching movies, playing games, or working — can interfere with sleep.
6. Try breathing techniques: Deep breathing is a relaxation technique that relieves tension and anxiety. Try inhaling slowly, holding for a few seconds, then exhaling slowly.

If these methods do not help you sleep, consider consulting a doctor or sleep specialist for further advice.

It is especially important to note that the default open-source ChatGLM-6B license forbids commercial use, but using it as a personal assistant is perfectly fine.

Chat-GLM-6B-INT4 is an INT4-quantized version requiring only about 4–6 GB of VRAM; most home computers with an 8 GB GPU can run it normally.

Yuanyu Intelligence's ChatYuan-large-v2

The first version of this model from Yuanyu Intelligence was briefly online when ChatGPT first appeared, but was taken down quickly for lacking a speech-control mechanism.

In any case, it remains a very good model, so a second generation has now been released. Its performance on Chinese Q&A and code generation is already quite strong, and a low-VRAM version is available — fully capable of serving as a personal computer assistant.

Beyond other tasks, its performance on math problems is also quite eye-catching:

Q: Compute 236588918+786777 and give the logical arithmetic rules
A: The number with the most digits is 236588918, with 9 digits. 786777 has 6 digits, fewer than the max, so pad zeros on the left to align: 000786777
236588918 from the right, digits 1 through 9 are 819825632
000786777 from the right, digits 1 through 9 are 777687000
Digit 1: 8 + 7 = 15, 15 ≥ 10, carry needed, write 5, carry 1
Digit 2: 1 + 7 + 1(carry) = 9, 9 < 10, no carry, write 9
Digit 3: 9 + 7 = 16, 16 ≥ 10, carry needed, write 6, carry 1
Digit 4: 8 + 6 + 1(carry) = 15, 15 ≥ 10, carry needed, write 5, carry 1
Digit 5: 8 + 8 + 1(carry) = 17, 17 ≥ 10, carry needed, write 7, carry 1
Digit 6: 5 + 7 + 1(carry) = 13, 13 ≥ 10, carry needed, write 3, carry 1
Digit 7: 6 + 0 + 1(carry) = 7, 7 < 10, no carry, write 7
Digit 8: 3 + 0 = 3, 3 < 10, no carry, write 3
Digit 9: 2 + 0 = 2, 2 < 10, no carry, write 2
Result: 237375695

Pay special attention: ChatYuan is Apache-2.0, but ChatYuan-large-v2's license is different — commercial use is not allowed; it may be used only for non-commercial research.

Worth noting is that ChatYuan-large-v2 uses the least VRAM of the bunch: with half-precision loading it needs only about 3 GB of VRAM to run.

It also provides a PaddlePaddle version, making further training on Paddle very convenient.

Summary:

These models are still some distance from ChatGPT, but in most situations they are already practical to use.

For non-commercial use, ChatYuan-large-v2 or Chat-GLM-6B are the preferred choices. For commercial use, you need to obtain authorization from the respective company.