Tag: linux

Using a BC250 crypto-mining board as an AI inference processor for local LLMs

I get empowered by being told something is impossible, or difficult, or too expensive, and this has been the case for me with trying to find an affordable way to run large language models at home.

I’ve still got a very expensive (something like AUD$2000) NVidia Jetson Orin NX 16GB that for the last couple of years has been my processor, but it is very slow in comparison to modern GPUs and dips just below the threshold of usable, especially with prefill sometimes taking 25 seconds before you get the first token out.

This is where a recent find enters the picture: the AMD BC250 crypto mining boards that can be bought for just over AUD$200 via AliExpress and other means and repurposed to be an AI processor (as well as games machine if you want).

This is the exact kind of thing that makes me excited: DIY methods of repurposing tech to do much more interesting things for cheap.

First up, as well as the card, you’re going to need a few things:
1. Power Supply (I chose a SilverStone DA750R Gold)
2. 2 x fans (I chose Noctua NF-A12x25 and a Noctua NF-A14x25 – it’s overkill but I wanted silence)
3. 1 x PWM Splitter (basically just splits the fan connector from the board to 2 x fans – mine is a Silverstone 100mm CPF01 black)
4. Cheap NVME SSD (I chose a Crucial E100 480GB drive, it doesn’t need to be fast as the board only supports older speeds)
5. Thermal paste (optional, but best to probably do this down the track as these cards have worked hard – I bought Frost X45)
6. Display port to HDMI adaptor (the board only has DP output and I have an HDMI screen)
7. USB Keyboard (for installing the OS)
8. USB thumb drive (for installing the OS)
9. Network cable (does not have wifi)
10. CR2032 button battery (for the BIOS/Clock settings)

Initial Assembly:
Place the board flat on a bench top sitting on something like matches to give it a tiny bit of air underneath. Install the SSD, Plug everything in and only install the largest of the 2 fans for now and just sit it on top of the heatsink (IMPORTANT YOU DO THIS). Plug the power supply in next to it but leave it switched off. Ensure that the PCIe cable from the power supply is connected to the J1000 connector with the 6+2 (8 total) connector, the other two power plugs next to it are unneeded. Lastly, don’t forget to install the CR2032 battery so that BIOS settings and clock can save reliably.

Power supply power on signal:
ATX power supplies don’t switch on unless they have 2 pins shorted (the motherboard usually does this work when you press your PC power button). This guide here will show you how:
https://www.youtube.com/watch?v=Ea1dcJ0QyAE
For now, just leave it as a small bit or wire or paperclip, but tape it up with electrical tape so it can’t come out or short on anything. In future it’ll be better to cut the wires and extract them to a switch or simply short them.

Preparing the USB thumb drive:
You’ll want to now download on your own computer Ubuntu Server 26.04 from the Ubuntu website:
https://ubuntu.com/download/server
You’ll also want to get Balena Etcher:
https://etcher.balena.io/

Using Balena Etcher, with your USB Thumb drive plugged in, choose the Ubuntu Server ISO you’ve downloaded, then choose the thumb drive, and Balena Etcher will set it up to be a bootable medium for the BC250.

Plug the USB into the BC250 in any of the open ports.

Turning on:
Ensuring that you have USB Keyboard and monitor plugged in, it’s time to finally turn on your board. Have one final check that you’ve got the correct jumper on the ATX power supply plug (wrong jumping can cause a failure/burn) and turn it on.

You should see the monitor spring to life, and BIOS come up asking you to set the time and initial settings. If this doesn’t happen, reboot to make sure you set this up.

Once done, hit save and exit, and hopefully watch your USB drive boot. Choose try/install from the menu and follow through the install. On mine I did a minimal install, no LVM, super basic everything to keep my footprint small (we want every last drop of RAM available).

On completion, it’ll ask you to remove the drive, and press enter. It should reboot and you now have a new system ready to go.

From here I recommend interacting with the board via SSH over network, so log in on the machine, and check its IP address with:
ip ad

On your local machine check you can connect to it via SSH at that address before you unplug the keyboard and monitor.

Configure base system packages/user config:
We’ll want to update the system packages as needed, and install some others to provide basic functionality.

Update/upgrade system:
sudo apt update
sudo apt upgrade

Install the build packages:
sudo apt install build-essential cmake git pkg-config

Mesa/Vulkan packages (for interacting with GPU):
sudo apt install mesa-utils vulkan-tools

Add your user to the system groups that are allowed to access the GPU:
sudo usermod -aG render,video $USER

Then reboot:
sudo reboot

Configure the GPU speed governor:
If we leave the GPU speeds to just sit at the default values, we’ll be using lots of power and generating lots of heat we don’t need to, so some people have dug into the speed governing of this quirky board.

You can see the source a tool for that here:
https://github.com/filippor/cyan-skillfish-governor

Simply follow the instructions on the page to install it (follow the building debian package instructions) and be aware you may need to install other packages along the way (if it tells you).

I’m writing this all after I’ve done the work, but I believe I had to install these:
sudo apt install cargo rustc libdrm-dev

I did set my minimum frequency to be 500mhz so I could really clock down when not in use, as AI inference workloads tend to be blob based, not continuous.

I could then see what speed the GPU was going with this:
watch -n 1 cat /sys/class/drm/card1/device/pp_dpm_sclk
(updates every 1 second, hit ctrl-c to quit it)

CPU Clocking:
You may think that we should do this also to the CPU speed, but as far as I can find and read, it doesn’t seem like this is something we can actually do. Some talk about it having a built in governor, so for now I didn’t go any deeper as some of the packages I found to address this said themselves that they didn’t do much.

VRAM/System RAM split:
The BC250 uses a shared RAM setup between system RAM and video RAM which isn’t unusual on semi-embedded boards/laptop systems. The trick is that we want to ensure that the system frees up as much as possible to the GPU so that we can run heavier LLM models.

For this, we will set what feels like a counter intuitive method of setting the GPU to 512 which is actually dynamic mode, so the more the GPU needs, it shuffles system ram down to compensate.

We will use this package to do that:
https://github.com/fanoush/bc250_memcfg

I did the following the build and set it:
cd ~
git clone https://github.com/fanoush/bc250_memcfg
cd bc250_memcfg
make sudo ./bc250memcfg UMA_SIZE 512 # 512 = dynamic, NOT a 512MB cap sudo reboot

The GTT ceiling fix:
The amdgpu driver defaults to capping GTT at roughly 50% of system RAM despite the dynamic setting we did above. We have to edit some kernel parameters in the bootloader (grub) so that the kernel behaviour with the driver changes.

Install nano text editor:
sudo apt install nano

Edit grub boot loader:
sudo nano /etc/default/grub

Change “GRUB_CMDLINE_LINUX_DEFAULT” to:
GRUB_CMDLINE_LINUX_DEFAULT="ttm.pages_limit=3959290 ttm.page_pool_size=3959290 amdgpu.gttsize=14750"
Press ctrl-o then enter to save, then ctrl-x to quit.

Update grub itself:
sudo update-grub

Reboot:
sudo reboot

Building Llama.cpp:
This is where the real AI inference happens. Llama.cpp is what we’ll use as our base. Before we start MAKE SURE YOU REALLY HAVE YOUR FAN ON THE HEATSINK: THIS WILL GENERATE LOTS OF HEAT!

Install the required packages:
sudo apt install libvulkan-dev glslc spirv-headers spirv-tools glslang-tools

Get the source code and build it:
cd ~
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp cmake -B build -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release cmake --build build --config Release -j$(nproc)

Confirm the GPU is detected as a real device, not falling back to CPU:
./build/bin/llama-cli --list-devices

Expect: Vulkan0: AMD BC-250 (RADV GFX1013) (XXXX MiB, XXXX MiB free)

And finally: test that it actually works (this will pull my favourite model from hugging face and run it with a small prompt:
./build/bin/llama-cli -hf unsloth/Qwen3.5-9B-GGUF:Q4_K_M -ngl 99 -p "Hello, introduce yourself briefly."

Running as a server for network links:
If you’re like me and you’ll want to link this with an agent framework (like my Supernova framework: https://github.com/JesseCake/supernova ) then you need to run LLama.cpp as an OpenAI compatible network endpoint.

This is the command I run mine with that fits well with the hardware (multiple slots for different contexts, tuning to run well on the BC250):
~/llama.cpp/build/bin/llama-server -hf unsloth/Qwen3.5-9B-GGUF:Q4_K_M --jinja -c 393216 --parallel 3 --flash-attn on --cache-type-k q4_0 --cache-type-v q4_0 --reasoning off --host 0.0.0.0 --port 8080 -b 8192 -ub 1024 --cache-ram 2048

Next up/Todo: TurboQuant version of LLama.cpp to provide speed and RAM improvements..

After that: Unlocking all 40 CU cores for even faster throughput

Prusaslicer not showing Plater window after background update

If you’re like me you woke up today with Prusaslicer having done an update in the background with Ubuntu’s Flatpak update mechanism hiding somewhere, and your Plater window showing nothing (just a blank area where the 3D field should be).

It took a bit of head scratching, but I realised that Flatpak has its own video drivers for Flatpak software, and these hadn’t been updated.

The fix was simple. Simply type:

flatpak update

In a terminal, and you’ll see something like:

Looking for updates…


        ID                                        Branch          Op      Remote       Download
 1. [ ] org.freedesktop.Platform.GL.default       24.08           i       flathub        1.0 kB / 145.4 MB
 2. [ ] org.freedesktop.Platform.GL.default       24.08extra      i       flathub        1.0 kB / 145.4 MB
 3. [ ] org.freedesktop.Platform.VAAPI.Intel      24.08           i       flathub        1.0 kB / 15.0 MB
 4. [ ] org.freedesktop.Platform.openh264         2.5.1           i       flathub      620.2 kB / 971.4 kB
 5. [ ] org.gnome.Platform.Locale                 48              i       flathub        1.0 kB / 394.5 MB

To which you press “y” to accept the updates. Bingo you should be back up and running.

Fix different scaling between multiple monitors on Ubuntu (22.04+)

If you’re like me, and use a laptop with a 4k screen that is actually far too fine to see, so you run it at 200% UI scaling, you’ll find it frustrating when plugging in external monitors.

This is also an annoyance across all operating systems really, but in Ubuntu, there is a quirk in which a change using the settings menu will apply the same scaling across ALL monitors meaning one monitor will be too small, the other too large.

The good news is there is a pretty simple fix.

At a command line (open terminal) type:

gsettings set org.gnome.mutter experimental-features "['scale-monitor-framebuffer']"

Then log out and in again, you can now individually change your monitor scaling to suit and no longer get frustrated with programs losing their minds when dragged between screens.

Fix Ubuntu 20.04 windows not appearing

I have a few old laptops kicking around that I keep running because with an SSD they’re perfectly fine to keep running. My 10 year old (approx) Samsung laptop is one of these.

Recently I did a fresh install of Ubuntu 20.04 onto this laptop and everything just worked – except for the fact that certain windows, including the Gnome Control Center (gnome-control-center) would open (showing the icon on the left bar) but seemed to quickly move off screen to the right.

I puzzled over this for quite some time trying all sorts of things including launching gnome-control-center from a terminal with -v flag set to see if something was wrong.

I stumbled across a few people talking about windows being off screen from 2014, and methods to bring them back so I tried the following:

  1. Open the control centre so that you see the icon on the left bar, and the red dot next to it showing that it’s running.
  2. Press alt-TAB until you see it highlighted so it’s definitely the window in focus
  3. Hold down alt-F7 and keep holding it, and tap the left arrow. Don’t release these keys yet
  4. Your mouse cursor should disappear now
  5. When you move your mouse left, you should see the window appearing into view
  6. Release alt-F7 and click your mouse when the window is in the middle of the screen

So this will bring the window back, but next time you launch it, it will disappear again. So there’s another problem going on here.

I choose to use the open source Nouveau Xorg video card drivers rather than the closed source, and often buggy on older machines, NVidia drivers, and this laptop has an NVidia card in it (Optimus era). It seems there is a problem, on this laptop at least, that it thinks there’s another display connected to the video output port on the card when I haven’t got anything plugged in.

So while we have the control panel on screen, we’ll go to the “Screen Display” section on the left side, and on the right we’ll choose “Single Display” up the top.

You should now no longer get windows launching off screen or seemingly not launching.