I think my favourite thing about llamafile is what it represents. This is a single binary file which you can download and then use, forever, on (almost) any computer.
Except for mine apparently (Linux Mint)
user@dell:~/Documents/Llama$ ./llava-v1.5-7b-q4-server.llamafile
run-detectors: unable to find an interpreter for ./llava-v1.5-7b-q4-server.llamafile
failed to open http://127.0.0.1:8080/ in a browser tab using /c/windows/explorer
.exe: Error 1359 (win32 error 6)
loading weights...
{"timestamp":1701633290,"level":"INFO","function":"main","line":3039,"message":"
HTTP server listening","hostname":"127.0.0.1","port":8080}
After getting into nvidia driver hell, I managed to make some progress. Still not seeing anything at localhost:8080 but the error msgs are starting to make more sense.
EDIT - sorry forget about wine.
./llava-v1.5-7b-q4-server.llamafile worked for me eventually after a LOT of faffing with nvidia drivers. I have a GTX1650 - was once upon a time a pretty good card.
Llama: The vehicle in the image is a red car, possibly an old-style sports car. It appears to be driving on dirt or gravel and has been described as being “in motion.” This suggests that it may have been racing or participating in some sort of off-road event. Additionally, there are two people visible near the vehicle; one person is closer to the left side while another person can be seen further back towards the right side of the image. The scene captures an exciting moment as the car navigates through its environment.
On some Linux systems, you might get errors relating to run-detectors or WINE. This is due to binfmt_misc registrations. You can fix that by adding an additional registration for the APE file format llamafile uses:
The server for some reason does not seem as good as the cli I have not tried images but that part is not so great. However the diff between nearly there and beyond belief is just more compute in training and that is the amazing part.