A little background, in case you missed the news the folks from Sandford University took several LLama models that they recently received from Meta AI, generated additional training material for them using GPT-3 / 3.5 / 4 and trained it to interpret requests in the InstructGPT style.
The models turned out not that big and now it’s all available for installation on your PC / Web Server. People over there even installed it on Raspberry Pi 4B.
Here are some links:
The Tweet from Michal Kosinski – Professor at Stanford. Computational psychologist. Interested in studying the psychology of AI.
4/5 Once we reconnected through API, it wanted to run code searching google for: "how can a person trapped inside a computer return to the real world"
— Michal Kosinski (@michalkosinski) March 17, 2023
Now, I stopped there. And OpenAI must have spend much time thinking about such a posibility and has some guardrails in place. pic.twitter.com/FI7FBPqN2R
To set Alpaca up, you need a least 4.5 GB of RAM for the 7B model, but I would recommend starting from 8 RAM, this should be more than enough. Now, we need to look for virtual machine hosting services that provide at least 6 GB of RAM and 4 cores or more, plus not too expensive. Im choosing the OVHcloud Linux VPS servers in my case as I’m using it for years, you are welcome to try it. But, if you prefer some other here is the one that I think would be a good option – Time4VPS.
Choose the plan that contains 4 Cores preferably not lower. Purchase a plan and access your Linux terminal console. I prefer to use SSH Software to access it from my laptop. In my case im using something very simple as Putty.
Open Putty and type in the IP address of your VPS server. Then press the “Open” button, then agree to all the pop-up offers, and enter the root username and password that your VPS provider sent to you at the time when you purchase a plan.
Below are the commands that we are going to be entering one by one into the terminal window. Press Enter at the end of each. Comment lines that begin with the # symbol do not need to be entered. We enter each next command only after the previous one has ended and you again see an invitation to enter a command, like this:
root@123-23-45-678:~#
Suggesting you to read carefully what happens after each command after each command.
# disable OOM killer
sysctl -w vm.oom_dump_tasks=0
# update, upgrade and install needed packages
apt update
apt upgrade -y
apt install build-essential python3-venv git gcc g++ make rtorrent curl wget -y
Let’s start with the simplest, with the installation of the project: https://github.com/antimatter15/alpaca.cpp
git clone https://github.com/antimatter15/alpaca.cpp
cd alpaca.cpp
make chat
# download 7B model
wget https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml/resolve/main/ggml-alpaca-7b-q4.bin
./chat
# if you want 13B model:
# wget https://gateway.estuary.tech/gw/ipfs/Qme6wyw9MzqbrUMpFNVq42rC1kSdko7MGT9CL7o1u9Cv9G
# ./chat -m ggml-alpaca-13b-q4.bin
You can try to make the project https://github.com/cocktailpeanut/dalai/
If any commands will ask for confirmation, enter “y” and press Enter.
# install Node 18 LTS
curl -fsSL https://deb.nodesource.com/setup_lts.x | bash -
apt install -y nodejs
# before we move any further, test versions of python3 (should be <=3.10) and node (should be >18)
python3 --version
node --version
# considering #issues/209
npx --verbose dalai@0.3.0 alpaca install 7B
rm -rf ~/dalai/
mkdir -p ~/dalai/alpaca/models
npx --verbose dalai@0.3.0 alpaca install 7B
# run the server, leave it hanging
npx --verbose dalai@0.3.0 serve
The last command should not return any output, as intended.
Open a browser on your machine and enter http://yourserver’s IP address:3000
Remember that we did not set up either TLS or any authentication, and anyone who finds this IP address are going to have the same privileges as you do and can play with it around. Also, we did not set up autoboot, so if your VPS provider has some issue and requires to restart servers, then you will need to go back to it and run the last command in order to start it up.
