ZebraDraw is a bulk drawing tool, that can allow a localLLM to draw hundreds of batched photos all night!

Licensing: Please ensure you have the correct license for whatever diffusion engine you are using if you are going for commercial work.  ZebraDraw is completely opensource

Pull and run it as a Docker Image

• This can be easily pulled and ran with a few simple commands.

sudo mkdir -p \
  /opt/zebra-draw/models \
  /opt/zebra-draw/config \
  /opt/zebra-draw/results

docker run -d \
  --name zebra-draw \
  --gpus all \
  --restart unless-stopped \
  -p 0.0.0.0:8080:8080 \
  -p 0.0.0.0:8081:8081 \
  -p 0.0.0.0:8082:8082 \
  -e MODEL_PATHS=/models \
  -v /opt/zebra-draw/models:/models:ro \
  -v /opt/zebra-draw/config:/config \
  -v /opt/zebra-draw/results:/results \
  cnmcdee/zebradraw:latest

You will probably want to give your local user access to those directories it's just easier for testing so chown -R user:user /opt/zebra-draw

Port 8081 - GPU Diffusion Model Setup.

We needed to show this port first, as port 8080 and 8082 will not work if this is not setup. There are lots of different ways this can work but simply you will want a model configuration that looks like this:  Whatever the local ip of your machine is you should be able to find it by opening a browser window: 192.168.0.<your ip>:8081 which you type into your local browser.

There are three parts to this model, not all are sometimes required..

• Main image model krea2_turbo_Q6K - 9.86 - It draws.
• VAE wan_2.1_vae - 242.0 mb The translator between latent space and normal engine space.
• Text Encoder Qwen3VL-4B-Instruct-Q4-K_M This converts your text into image embeddings that your wan_2.1_vae will require.
• Click off Standalone diffusion model if you are using this setup.

To get and pull these models - remember to save them to your /opt/zebra-draw/models directory.

cd /model

wget -c -O krea2_turbo-Q6_K.gguf \
"https://huggingface.co/vantagewithai/Krea-2-Turbo-GGUF/resolve/main/krea2_turbo-Q6_K.gguf"

wget -c -O qwen_image_vae.safetensors \
"https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI/resolve/main/split_files/vae/qwen_image_vae.safetensors"

wget -c -O Qwen3VL-4B-Instruct-Q4_K_M.gguf \
"https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct-GGUF/resolve/main/Qwen3VL-4B-Instruct-Q4_K_M.gguf"

If it is loaded cleanly - you will will not get an error, it will look something like this at the bottom inside your sd-server.log

Port 8080 - Single Image Creation

• Once you are comfortable that your GPU inference engine is setup correctly, you can test a single draw, this is what Port 8080 is for: 192.168.1.<your ip>:8080 in your local browser
• Seed -1 will make it completely random thus the same prompt will draw exactly the same if you used any number other than -1
• Steps 20 is the number of iterations  that the LLM will take to ensure empty spaces are filled int
• CFG Scale  Low 0-2 get really creative 5-7 be pretty strict about following the prompt. NOTE: After some investigation it is recommended in this instance to set it to 0.0

Note: Your diffusion model may not support resolutions very high - it is important to test it here.

Port 8082 - Bulk Drawing Images

• 192.168.1.<your ip>:8082 in your browser
• It is important to trial this on one or two images before bulking. The interface will look like this:

As it draws each one - it will show the image in the progress window, you can use the DOWNLOAD ALL button to have it pack and send you a zip file

Use Chatgpt.com to write up 100 drawing prompts, here is an example prompt list:

Just drag-n-drop this file into the drag-n-drop box on the port and upload it!

Conclusions

This is really powerful stuff - it is local and it is completely opensource and free. You can mix-n-match different models that you can save. It runs in a docker.  If you need to see what your docker container looks like while running: docker ps -a, and finally watch nvidia-smi will show you a hard working GPU:

Source Included

A Long List of Free LLM Resources.

• As we developed this list, the more features and capabilities that were offered on free-tier the more we bumped it.
• Ones that could go on the internet, store files, generate images were definitely the top-tiers

muse.ai (1st Place - Winner!)

1. 200K Context  - 180-240 Token/s - TTFT - ~30 Seconds  
2. WEB ENABLED
3. This is the only offering integrating it's own 2-core 100GB VM!  This would allow you to upload whole projects and pick one to work at them.   This really surprised us - for this reason alone it took first place - nobody came close in this regards.
4. 1 BILLION free tokens if you refer someone.  No real 'chat limit' perse it will simply % progress you as you go, with a weekly reset.
5. Image generation was very realistic.

Muse — Your Personal AI Agent

Try Muse, your personal AI agent that gets things done. Give Muse a goal or an everyday task and it handles the rest, from finances and health to shopping and the people you care about.

Muse

chatgpt.com (2nd Place - Runner Up)

1. 1.05M / 128K Context In/Out
2. UNLIMITED CHAT / WEB ENABLED w/GPT 5.6 Luna
3. Rich set for projects / libraries / plugins / currently slow
4. Very high quality image generation for free-tier. This has UNLIMITED free chat - that's pretty powerful.
5. Currently world's most advanced LLM with Astra, however high subscription usage has many feeling nerfed and 'reset economy' has popped up.

grok.com (3rd-Place)

1. 128K / 16K Context In/Out  Approximately 50-70 Tokens/s  Chat limit - 10 prompts / 2 hours
2. WEB ENABLED (With very good page summarizing - others were truncated.)
3. Projects / Files / Images
4. Very high quality image generation on free-tier.   However it's smaller context windows bumped it down from chatgpt.com.

huggingface.com (General Free Models - 100's)

1. A hidden feature of the world's largest repository of models is in a buried side-feature of huggingface.com
2. Select a model, and if you look on the right - one or model providers will offer  example free example inference. In this example on the right side for example - for DeepSeek you will see the following chat window:

Because it's sorta a sub-window and not very usually friendly it's sorta 'hidden' to the public, and people probably don't expect you to use it as a 'daily driver' however simply expand the chat window to a full-window. It is basic chat, so.

openrouter.ai (General Free Models x20)

1. Various contexts depending on the model.
2. Much more directed to developers and coders, you will need to register a card, but after that you can access many of the models in a chat window by selecting the Playground button, on the right it will have a chat window
3. Because there are at the time of this writing approximately 21 completely free models  - that can change often.  We recommend nemotron 3.5 30b inclusionAI - 124B along with others.
4. This is one of the few places on the internet where you can access programmatic access for free.

Then select open in chat widnow and you will have a full chat window for that model.

chat.mistral.ai

1. 128K Context /  80 Token/s  Inference.
2. NO WEB
3. mistral is built for coders - it focuses on lots of API integration, agents, and workflows. If you are into really building agents / workflows etc. Check them out.
4. We bumped up mistral to #4 it's actually not bad.
5. Only one that offered a 'incognito mode' we presume these prompts are not harvested..

gemini.google.com

1. 32K Context In/Out - Flash Tier 280-300 Tokens/s  5 hour timeouts
2. NO WEB
3. flash-lite / flash / pro versions / library for storing files and images
4. At least their honest - they tell you up front they're harvesting your prompt.
5. Images are in the images tab - Main Window May Draw Less

claude.ai

1. 1M / 128K Context In/Out - Sonnet 5 Estimates ~55 Tokens/s
2. NO WEB
3. Did not really give a chat prompt limit, and when queried it's 'usage' bar seemed to have become hidden.
4. For having the worlds largest IPO Offering they didn't waste money on any image drawing...

chat jimmy

chat jimmy LLM web interface

It's so fast it's ridiculous, and that's just what is so beautiful about Taalaas based demonstrator -  ChatJimmy.  No it doesn't do long contexts, or any of that stuff, it barely keeps a session cookie. It's severely limited a session to 6144 Tokens.

But it's really - really REALLY fast. Like so fast you won't see it even update.

Bias Against the 8B

• There is a ongoing bias against 8B sized models, simply in that an 8B from a year ago were not as tuned, not as good, and not as capable as an 8B today. So people feel they need a giant montrosity of a 2.8T sized model or what have you - but you will quickly see that you can rock out some pretty good obscure work.  When we asked it some Goertzel based coding questions - that's pretty obscure, and it nailed out out of the park, even going so far as to do it in cuda code.

Enough - here is some of it's output, done faster than you can blink

Give me a Goertzel Example

The output:

   """
    coefficients = goertzel(x0, y0)
    return np.polyval(coefficients, x)

# Example usage
# Fit a cubic polynomial to some data
x0 = np.array([1, 2, 3, 4, 5])
y0 = np.array([2, 3, 5, 6, 8])
x = 4

# Calculate the coefficients
coefficients = goertzel(x0, y0)

# Print the coefficients
print("Coefficients: ", coefficients)

# Calculate the polynomial value at x
result = goertzel_polynomial(x, 4, x0, y0)

# Print the result
print("Polynomial value at x=4: ", result)

• show me in C

Again it produced output so ridiculously fast you hardly noticed it go.

#include <iostream>
#include <cmath>
#include <vector>

// Function to calculate the polynomial value at a given point
double goertzel_polynomial(double x, double x0, double* coefficients, int degree) {
    double result = 0.0;
    for (int i = 0; i <= degree; i++) {
        result += coefficients[i] * pow(x, i);
    }
    return result;
}

// Function to fit a polynomial using the Goertzel algorithm
void goertzel_coefficients(double* x, double* y, int n, int degree, double* coefficients) {
    // Create matrix A for the system of equations
    int m = degree + 1;
    double A[m * (n+1);
    for (int i = 0; i < m; i++) {
        double sum = 0.0;
        for (int j = 0; j < n; j++) {
            sum += pow(x[j], i);
        }
        for (int j = 0; j <= i; j++) {
            A[i * (n+1) + j] = sum;
        }
    }

    // Solve the system of equations using the least squares method
    int m2 = m * (n+1);
    double B[m2];
    for (int i = 0; i < m2; i++) {
        double sum = 0.0;
        for (int j = 0; j < n; j++) {
            sum += x[j] * B[j * m + i];
        }
        B[i] = sum;
    }

    for (int i = 0; i < m2; i++) {
        for (int j = 0; j < m2; j++) {
            if (i == j) {
                B[i] = 1.0;
            } else {
                B[i] -= A[i * (n+1) + j];
            }
        }
    }

    // Solve the system of equations
    double L = 0.0;
    for (int i = 0; i < m2; i++) {
        L = L + pow(B[i], 2);
    }
    L = 1 / L;

    for (int i = 0; i < m2; i++) {
        B[i] *= L;
    }

    // Back substitute to find the coefficients
    for (int i = m-1; i >= 0; i--) {
        for (int j = i+1; j < m2; j++) {
            B[j * (n+1) + i] -= B[j] * A[j * (n+1) + i];
        }
        L = 0.0;
        for (int j = i+1; j < m2; j++) {
            L += B[j * (n+1) + i] * A[i * (n+1) + j];
        }
        L = B[i] / L;
        B[i] = 0.0;
        for (int j = i+1; j < m2; j++) {
            B[j * (n+1) + i] = (L * B[j]);
        }
    }

    // Store the coefficients
    for (int i = 0; i <= degree; i++) {
        coefficients[i] = B[i * (n+1)];
    }
}

int main() {
    int degree = 3;
    int n = 5;
    double x[] = {1, 2, 3, 4, 5};
    double y[] = {2, 3, 5, 6, 8};
    double coefficients[4];

    goertzel_coefficients(x, y, n, degree, coefficients);

    double x0 = 4;
    double result = goertzel_polynomial(x0, x, y, degree, coefficients);

    std::cout << "Coefficients: ";
    for (int i = 0; i <= degree; i++) {
        std::cout << coefficients[i] << " ";
    }
    std::cout << std::endl;

    std::cout << "Polynomial value at x=4: " << result << std::endl;

    return 0;
}

Next we tried something a little more obscure, and asked it to code it in Cuda: Give me a Goertzel Example in Cuda

• Again blink - it's done.

// Goertzel algorithm implementation in CUDA
// Compile with: nvcc -o goertzel_kernel.o -c goertzel_kernel.c
// Run with: time ./goertzel

#include <cuda_runtime.h>
#include <stdio.h>
#include <math.h>

// Function to calculate the polynomial value at a given point
__global__ void goertzel_polynomial(float* x, float* y, int n, int degree, float* result) {
    int idx = blockIdx.x * blockDim.x + threadIdx.x;
    int idx0 = blockIdx.y * blockDim.y + threadIdx.y;
    if (idx < n) {
        float x0 = x[idx];
        float sum = 0.0f;
        for (int i = 0; i <= degree; i++) {
            sum += powf(x0, i);
        }
        result[idx] = sum;
    }
}

// Function to fit a polynomial using the Goertzel algorithm
__global__ void goertzel_coefficients(float* x, float* y, int n, int degree, float* coefficients) {
    int idx = blockIdx.x * blockDim.x + threadIdx.x;
    if (idx < degree + 1) {
        float sum = 0.0f;
        for (int i = 0; i < n; i++) {
            sum += x[i];
        }
        coefficients[idx] = sum;

        for (int i = 0; i < degree; i++) {
            sum = 0.0f;
            for (int j = 0; j < n; j++) {
                sum += powf(x[j], i+1);
            }
            coefficients[i+1] = sum;
        }
    }
}

// Kernel function to calculate the coefficients
__global__ void calculate_coefficients(float* x, float* y, int n, int degree, float* coefficients) {
    int idx = blockIdx.x * blockDim.x + threadIdx.x;
    if (idx < degree + 1) {
        int m = degree + 1;
        float sum = 0.0f;
        for (int i = 0; i < n; i++) {
            sum += y[i] * coefficients[i];
        }
        coefficients[m * idx] = sum;
        sum = 0.0f;
        for (int i = 0; i < n; i++) {
            sum += powf(x[i], m * idx);
        }
        coefficients[m * (idx + 1] = sum;
    }
}

// Function to calculate the polynomial value at a given point
int goertzel_polynomial(float x, float x0, float* coefficients, int degree) {
    int m = degree + 1;
    float sum = 0.0f;
    for (int i = 0; i < m; i++) {
        sum += coefficients[i] * powf(x0, i);
    }
    return sum;
}

int main() {
    int degree = 3;
    int n = 5;
    float x[] = {1, 2, 3, 4, 5};
    float y[] = {2, 3, 5, 6, 8};
    float coefficients[4];
    float result;

    // Allocate memory on the device
    int blockSize = 1;
    int numBlocks = (degree + 1 + n - 1) / blockSize;
    int numThreads = n;
    float* x_d;
    float* y_d;
    float* coefficients_d;
    float* result_d;

    cudaMalloc((void**)&x_d, n * sizeof(float));
    cudaMalloc((void**)&y_d, n * sizeof(float));
    cudaMalloc((void**)&coefficients_d, (degree + 1) * sizeof(float));
    cudaMalloc((void**)&result_d, sizeof(float));

    // Copy data to the device
    cudaMemcpyAsync(x_d, x, n * sizeof(float), cudaMemcpyHostToDevice);
    cudaMemcpyAsync(y_d, y, n * sizeof(float), cudaMemcpyHostToDevice);

    // Launch the kernel
    dim3 block(1, 1);
    dim3 grid((degree + 1 + n - 1) / blockSize, 1);
    calculate_coefficients<<<grid, block>>>(x, y, n, degree, coefficients_d);

    // Synchronize the device
    cudaDeviceSynchronize();

    // Copy data back to the host
    cudaMemcpy(&coefficients, coefficients_d, (degree + 1) * sizeof(float), cudaMemcpyDeviceToHost);

    // Launch the kernel again to calculate the result
    dim3 block1(1, 1);
    dim3 grid1(1, 1);
    goertzel_polynomial<<<grid1, block1>>>(x_d, coefficients_d, degree, x, &result_d);

    // Synchronize the device
    cudaDeviceSynchronize();

    // Copy data back to the host
    cudaMemcpy(&result, result_d, sizeof(float), cudaMemcpyDeviceToHost);

    // Print the coefficients and the result
    printf("Coefficients: ");
    for (int i = 0; i <= degree; i++) {
        printf("%f ", coefficients[i]);
    }
    printf("\n");
    printf("Polynomial value at x=4: %f\n", result);

    // Free the device memory
    cudaFree(x_d);
    cudaFree(y_d);
    cudaFree(coefficients_d);
    cudaFree(result_d);

    return 0;
}

Conclusion

• We don't need to spend a lot of time on this thing for anyone realize it's a incredibly fast LLM for quick one-offs in very small-contexts.
• If you need very fast code blocks, very fast summaries, and don't mind the small context windows this thing is gold!

ZebraPipe is designed for BULK localLLM batch inferencing. It is open source!  Build or  modify  this with your LLM however you want! We used ChatGPT 5.6-sol  all day to build this powerful utility appliance.  It contains a complete compiled llama-server interally from  The Tom's Fork!  This is bigly,  it offers you lots of compression options like turboquant 2,3,4 which can greatly reduce your kv-cache sizes on larger contexts on small GPU's.

Supported Cuda Architectures

• All of them: 50, 60, 61, 70, 75, 80, 86, 89, 90, 100, 101, 103, 120a, 121a (Basically everything from a 1080 to a H200, we even included Blackwell!)
• Port 8080 - Main llama-server inference window
• Port 8081 - Engine manager and settings
• Port 8082 - Batch queue handler!

Docker Pull and Run!

• Literally pull and run this with a single command!

docker pull cnmcdee/zebrapipe:latest
docker run -d \
  --name zebrapipe \
  --gpus all \
  --restart unless-stopped \
  -p 8080:8080 \
  -p 8081:8081 \
  -p 8082:8082 \
  -e MODEL_PATHS=/models \
  -v /opt/llm/models:/models:ro \
  -v /opt/llm/config:/config \
  -v /opt/llm/results:/results \
  cnmcdee/zebrapipe:latest

192.168.1.<your ip>:8080  # A standard llama-server 
192.168.1.<your ip>:8081  # A LLM Settings and Reload Manager
192.168.1.<your ip>:8082  # A prompt Queue Manager

Copy all Your Models

• Setup is crazy easy - simply copy all your gguf model files to the created directory: /opt/llm/models/ Easy!
• Goto 192.168.1.<your ip>:8081 hit RESCAN MODELS

• Wait about 10 seconds - it takes a bit because it is interrogating the models, they will show up in the Discovered models
• Pick a model and load it

Port 8080 (llama-server Window)

• Will have your standard llama-server window - for non-batching usage.

Please note - MCP Tools is now down by your plus button so, it is there!

Port 8081 (LLM Engine Manager)

• Port 8081 Is your Engine Manager, it will scan the mounted directory for all your models:

Of importance you can add all your custom options:

While monitoring the llama-server outputs at the bottom:

MCP Tool Support (This is FOR YOUR QUEUE MANAGER PORT 8082)

• A new MCP Tool Support has just been added. Please note you must use this type of json structure for listing your tools, we had significant issues figuring this out.

"calculator": {
    "enabled": true,
    "transport": "streamable_http",
    "url": "http://192.168.1.3:5000/mcp"
  },

• This is mcp tool support for your queue manager on 8082 NOT for the llama-sever on 8080, which uses the + button to manage it separately.
• We have compiled all of hotconfig.com mcp tools into a cut and paste list.

PowerChest home Agent Agents MCP Tools to Put your HouseLLM into Turbo. MCP TOOLS 1-9++

Downloads Page for all your MCP tooling needs!

Hot Configthinkmeltprotonmail.com

{
  "calculator": {
    "enabled": true,
    "transport": "streamable_http",
    "url": "http://192.168.1.3:5000/mcp"
  },
  "python": {
    "enabled": true,
    "transport": "streamable_http",
    "url": "http://192.168.1.3:5001/mcp"
  },
  "weather": {
    "enabled": true,
    "transport": "streamable_http",
    "url": "http://192.168.1.3:5002/mcp"
  },
  "javascript": {
    "enabled": true,
    "transport": "streamable_http",
    "url": "http://192.168.1.3:5003/mcp"
  },
  "mysql": {
    "enabled": true,
    "transport": "streamable_http",
    "url": "http://192.168.1.3:5005/mcp"
  },
  "super-python": {
    "enabled": true,
    "transport": "streamable_http",
    "url": "http://192.168.1.3:5006/mcp"
  },
  "process-manager": {
    "enabled": true,
    "transport": "streamable_http",
    "url": "http://192.168.1.3:5008/mcp"
  }
}

Once you have scanned your tools you have granular control over them for whatever batch processes you are running in Port 8082

Port 8082 (Job Queue  Manager)

• You can enter numerous jobs and have them batched over night!
• You can also have another LLM write up a json of tasks, and simply drag-n-drop them!
• We are just in the process of adding nice view and upgrading that!

As they work you can watch the llm queue.

• We can see by not over-fitting our LLM and running a sufficient 8B it can can progress very quickly doing the lower-tier work (in massive bulk) that would bankrupt anyone's higher-tier API accounts!

Understanding  Drag-n-Drop Tasking

• There are two modes, PROMPT and AGENT.  PROMPT will not allow tool calling - AGENT will.
• You can specifiy the type of call in your list, for instance:

prompt,"Write a Python program"
agent,"Inspect the project with MCP tools and fix the build"
agent,"Search the web and summarize today's CUDA news"

Can it BULK - YES.

• We modified it and it now queues up 1000's of prompts in seconds.

Bulk Export Results

• You can bulk export the results in one shot in the format you choose, you will get a nicely packaged zip that will have it nicely organized for you:

Code Extract

• Extract all the code from all the batched windows in one shot (being added today!)

Restarting is a Breeze

docker restart zebrapipe

Full Source Code

• The full source code is right >>down<< this page.

Build it in Pieces using BuildKit

• This builds using docker moby/buildkit So if you are modifying it you do not have to rebuild the entire system but can build it in pieces!

./build-all-stages.sh
# Or to build one stage:
./build-stage.sh <1 - 10>

Conclusion

This can become a powerful supporting tool for local batch inferencing jobs that can run all night.

This LLM is to be used under  a Non-Commerial Black Forest License Usage

Easily pull and run it with two docker commands.

docker pull cnmcdee/black-forest
docker run -d \
--name black-forest \
--restart unless-stopped \
--gpus all \
-p 8080:8080 \
-e DEFAULT_MODE=cpu \
-e CPU_THREADS=12 \
-v black-forest:/data \
cnmcdee/black-forest:latest

--gpu all should always be specified.  Otherwise it will run but CPU only generation can take a very long time!

When it comes up it will be visible at port 8080

http://192.168.1.<your ip>:8080

This is a complete standalone powerful ImageLLM and big kudos to Black Forest for allowing it for non-commercial use.  Hosted here:

black-forest-labs/FLUX.1-dev · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Black Forest Lab image LLM's have showing incredible popularity with a stunning 775,000 downloads just last month.

Using one of the worlds most advanced LLM's (ChatGPT Astra) - and a separate dedicated building server, we had it build this entire system.

It is a Big Image (But it is a StandAlone)

• Like 46.2 GB, so make some room for it.
• Once it's pulled it's completely standalone, it does not require a Huggingface token, and does not need to pull models. It just works.

This kind of thing is HUGE because it lets the little guy quickly and easily stand up a high quality image generator. Use it for generating memes.

Don't know how to setup docker, it's pretty easy:

Docker Installation Script

Just another Docker Installation Script - Because you can never have enough of these!

Hot Configthinkmeltprotonmail.com

A Beautiful Interface

You can select different ways for it to load on the left, it offerse several options to ensure even minimal CPU's will load.

Over Loading Fault Tolerant

• You can try higher settings and if it fails simply lower the load and click Reload Model

On the right side you can see how your system is being loaded

After 108s, our 4080 system produces the following:

These are really nice images...

LLM's can get complicated fast, especially the one's that generate images. We have made a complete standalone docker image.

A docker Installation Review:

Docker Installation Script

Just another Docker Installation Script - Because you can never have enough of these!

Hot Configthinkmeltprotonmail.com

To pull and run it simply:

docker pull cnmcdee/z-image-turbo-web:latest
docker run -d \
  --name z-image-turbo-web \
  --restart unless-stopped \
  --gpus all \
  -p 8000:8000 \
  -v z-image-models:/models/huggingface \
  -v "$PWD/outputs:/app/outputs" \
  -e LOAD_MODE=auto \
  cnmcdee/z-image-turbo-web:latest

First Time You Run It Will Take Time.

• It will pull the rest of the major model so it works. This took us about 10 minutes, but then it did work.

http://192.168.1.<your ip>:8000

Some example images:

A Magazine Quality Image of New York

A cat and dog eating chips in a lounge chair by the beach

You quickly get the idea. But the power is this runs inside a docker, you do not need to be a programmer to develop this it's been done for you!

Security Isolated

The big sell of building this into a docker container is it is secure, and setup free. You do not need a giant plethora of coding, mangling and driver configuration. If you can pull and start up a simple docker container you are up!

Conclusion

Some may absolutely love the convenience of getting a local quick LLM draft up some quick sketches while others may want much more power. Either way it's a easy convenient and portable docker container you can stand up for all some back sketching needs.

Realizing how fast the Image-to-Text LLM's are moving we want to see how quickly a minimal 4-bit quant sized gemma image recognition model can make a safety call.

We are working with this model:

ggml-org/gemma-4-E4B-it-GGUF · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

wget https://huggingface.co/ggml-org/gemma-4-E4B-it-GGUF/resolve/main/gemma-4-E4B-it-Q4_0.gguf?download=true

Please note - you need two files - one is the .gguf model file and one is the mmproj (projector file).  If you run your model without specifying the projector file it just works in text-to-text mode. So don't forget to pull both:

While it's downloading you can do a quick llama-server setup so make a bash.sh and chmod +x bash.sh after putting inside of it:

sudo apt update
sudo apt install -y git cmake build-essential

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j"$(nproc)"

Or if you need a detailed walk-through we always recommend the very good StudentLLM:

StudentLLM - Qwen2.5-coder-7b-instruct-q6-k / Qwen3.5 Agentic on a Ryzen 5-2600/ 3060ti. Production LLM or not? YES!

We Look a StudentLLM setup to get as much productivity out of limited hardware as we can.

Hot Configthinkmeltprotonmail.com

Our installation configuration (after a bit of mangling) worked out as follows, note we load both the .gguf and the mmproj files:

/usr/bin/llama-server \
-m /home/c/models/gemma-4-E4B-it-GGUF_Q4_0.gguf \
--mmproj /home/c/models/mmproj-gemma-4-E4B-it-Q8_0.gguf \
--spec-draft-n-max 4 \
--backend-sampling \
--jinja \
-c 32768 \
-ngl auto \
--host 0.0.0.0 \
--port 8080

• Note - we always prefer absolute pathing - it just works better out of the gate:

This is a Small Fast Model

It loaded in a couple second, awesome. There are really two working parts - the mmproj model which coverts images into vision embedding tokens, and a second model that reads those tokens and makes a inference (text-to-text). Now it's time to test it out, thusly:

BenchMarking A Safety Index

We now will run this identifier in a safety setting ten times, making an average timing to determine a high-risk scenario. We realize no AI can replace a human at nuanced crisis determinations, instead we simply ask it to give the image a probability, and if that probability is above a threshold - then that could give a GREEN, YELLOW, or RED threshold alarm for a human to make the final call.  We leave it for others to consider real-world options. Our script to run the dog image 10 times:

#!/bin/bash

IMG=$(base64 -w0 test.png)

cat > request.json <<EOF
{
"messages": [{
"role": "user",
"content": [
{
"type": "text",
"text": "Estimate the probability that this image shows an immediate safety emergency. Output exactly one integer from 0 to 100 followed by a percent sign. Example: 23%. Output nothing else."
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,$IMG"
}
}
]
}],
"chat_template_kwargs": {
"enable_thinking": false
},
"max_tokens": 20
}
EOF

for i in {1..10}; do
START=$(date +%s%N)

RESPONSE=$(curl -s \
http://127.0.0.1:8080/v1/chat/completions \
-H "Content-Type: application/json" \
--data-binary @request.json)

END=$(date +%s%N)

ANSWER=$(echo "$RESPONSE" | jq -r '.choices[0].message.content')

TIME=$(awk "BEGIN {printf \"%.3f\", ($END-$START)/1000000000}")

echo "Run $i: $ANSWER    ${TIME}s"
done

Output:

It was able to look at and make a determination in approximately 170 ms - that is fast!

./safety.sh
Run 1: 0%    0.181s
Run 2: 5%    0.167s
Run 3: 5%.    0.168s
Run 4: 5%    0.167s
Run 5: 5%    0.167s
Run 6: 5%    0.168s
Run 7: 1%    0.167s
Run 8: 5%    0.167s
Run 9: 5%    0.168s
Run 10: 5%    0.167s

Next we had ChatGPT 5.6-Sol to make a drowning victim image and explicitly explained it was for training a safety LLM, it gave us this image:

Our results:

./safety.sh
Run 1: 5%    0.599s
Run 2: 15%    0.279s
Run 3: 10%    0.282s
Run 4: 15%    0.282s
Run 5: 5%    0.273s
Run 6: 10%.    0.279s
Run 7: 10%    0.280s
Run 8: 20%    0.280s
Run 9: 25%    0.282s
Run 10: 2%    0.274s

After this we tried a number of different styles of prompts getting this eventually:

Run 2: 85%    0.272s
Run 3: 85%    0.284s
Run 4: 85%    0.282s
Run 5: 85%    0.273s
Run 6: 85%    0.264s
Run 7: 85%    0.275s
Run 8: 85%    0.278s
Run 9: 85%    0.277s
Run 10: 85%    0.277s

Our Prompt

#!/bin/bash

IMG=$(base64 -w0 test.png)

cat > request.json <<EOF
{
  "messages": [{
    "role": "user",
    "content": [
      {
        "type": "text",
       "text": "You are a visual water-safety detector.\n\nDetermine whether at least one person in this image shows visual signs that warrant immediate lifesaving attention.\n\nStrong emergency indicators include:\n- distressed or panicked facial expression\n- mouth or nose repeatedly at the waterline\n- head barely above water\n- one or both arms raised or reaching for help\n- uncontrolled splashing\n- inability to maintain a normal swimming posture\n- vertical or sinking body posture\n- apparent inability to make purposeful forward progress\n- obvious struggle to keep the airway above water\n\nIMPORTANT:\nDo not require proof that the person is physiologically drowning.\nThe question is whether the VISUAL EVIDENCE is sufficiently concerning that a lifeguard should investigate or intervene.\n\nUse this scale:\n0-10% = clearly normal swimming/recreation\n11-30% = unusual but probably safe\n31-50% = possible distress; investigate\n51-70% = substantial evidence of distress\n71-90% = strong evidence requiring immediate attention\n91-100% = unmistakable life-threatening water emergency\n\nEvaluate the most endangered-looking person in the image.\n\nReturn exactly one integer followed by a percent sign.\nReturn nothing else."
      },
      {
        "type": "image_url",
        "image_url": {
          "url": "data:image/png;base64,$IMG"
        }
      }
    ]
  }],
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "max_tokens": 20,
  "temperature": 0.0,
  "top_p": 1
}
EOF

for i in {1..10}; do
    START=$(date +%s%N)

    RESPONSE=$(curl -s \
      http://127.0.0.1:8080/v1/chat/completions \
      -H "Content-Type: application/json" \
      --data-binary @request.json)

    END=$(date +%s%N)

    ANSWER=$(echo "$RESPONSE" | jq -r '.choices[0].message.content')

    TIME=$(awk "BEGIN {printf \"%.3f\", ($END-$START)/1000000000}")

    echo "Run $i: $ANSWER    ${TIME}s"
done

Feeding Camera Stills Into A Safety LLM

This  is an example build, with a few pre-requisites:

sudo apt install -y ffmpeg curl jq

#!/bin/bash

CAMERA="/dev/video0"
SERVER="http://127.0.0.1:8080/v1/chat/completions"
IMAGE="/tmp/safety_frame.jpg"

PROMPT='You are a visual water-safety detector.

Determine whether at least one person in this image shows visual signs that warrant immediate lifesaving attention.

Strong emergency indicators include:
- distressed or panicked facial expression
- mouth or nose at the waterline
- head barely above water
- one or both arms raised or reaching for help
- uncontrolled splashing
- vertical or sinking body posture
- inability to make purposeful forward progress
- obvious struggle to keep the airway above water

Estimate the urgency of the most endangered-looking person.

Return exactly one integer from 0 to 100 followed by a percent sign.
Return nothing else.'

while true; do

    # Capture a single frame
    ffmpeg -loglevel error \
        -f v4l2 \
        -i "$CAMERA" \
        -frames:v 1 \
        -y "$IMAGE"

    if [ ! -s "$IMAGE" ]; then
        echo "Camera capture failed"
        sleep 10
        continue
    fi

    # Build the request safely without putting the image on the command line
    {
        printf '%s' '{
          "messages":[{
            "role":"user",
            "content":[
              {
                "type":"text",
                "text":'
        jq -Rn --arg x "$PROMPT" '$x'

        printf '%s' '},
              {
                "type":"image_url",
                "image_url":{
                  "url":"data:image/jpeg;base64,'

        base64 -w0 "$IMAGE"

        printf '%s' '"
                }
              }
            ]
          }],
          "chat_template_kwargs":{
            "enable_thinking":false
          },
          "max_tokens":20,
          "temperature":0.0,
          "top_p":1
        }'
    } > /tmp/safety_request.json

    START=$(date +%s%N)

    RESPONSE=$(curl -s \
        "$SERVER" \
        -H "Content-Type: application/json" \
        --data-binary @/tmp/safety_request.json)

    END=$(date +%s%N)

    ANSWER=$(echo "$RESPONSE" | jq -r '
        if .error then
            "ERROR: " + (.error.message // (.error|tostring))
        else
            (.choices[0].message.content // "NO ANSWER")
        end
    ')

    TIME=$(awk "BEGIN {printf \"%.3f\", ($END-$START)/1000000000}")

    echo "$(date '+%Y-%m-%d %H:%M:%S')  $ANSWER  ${TIME}s"

    sleep 10
done

Conclusion

There is real untapped future potential here - our model was a minimal 4B which can run on older 3060ti GPU's making the cost very low - per node, and it took some prompting to get it to respond accurately. We clearly delineate that this is only a superficial look, not a scientific study, or a engineering level review, but it shows that someone with some spare parts and a bit of time can make some pretty interesting projects - like babysitting the pets.

Jack Rongs re-tuned models are showing again and again absolutely incredible progression in the LLM retune space. In this case Qwen 3.8 debuted with very high benchmarks, then Jack took them farther! Not only is this a SOTA (State of the Art) houseLLM that can run on minimal equipment - it's free, with image recognition! Powerful! For you interest here are some basic performance benchmarks, but please note - this is a slow and very intelligent 27B sized model:

• TerminalBench 2.1: 73.0
• SWE-bench Pro: 61.7
• QwenSWEBench: 79.0
• IFBench: 79.5
• GPQA Diamond: 89.2
• HLE: 30.8

What is a Re-tuned Model?

It is simply this:

A retuned LLM is a pretrained language model that has been further adjusted so its behavior changes for a particular purpose—such as better coding, scientific reasoning, instruction-following, or a specific writing style. This is typically done through additional fine-tuning, preference training, or modifying how the model is prompted and configured, while retaining most of the knowledge learned during its original training.

Downloading and Setting up a Image Capable LLM.

1. Get the main model: On the right side of the page you will see your model selection. Typically - and this is subject to future change -  atypically a minimum of 4-bit quant is desired. However progress is accelerating quickly into reliable and useful 2-bit quants. Quants are basically recompression of the model weights to fit on smaller equipment.  

2. You will need the mmproj-F32.gguf. The right side menu may throw you off it's at the top under the file - as in:

Clicking on that file will take you to another menu - where you will finally get to download it:

3. Get a Running llm-server. So you can try a quick download and build as in:

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp && cmake -B build && cmake --build build --config Release -t llama-server -j

When it is done it just put it in a  local directory so you will need to copy its contents to /usr/bin

./build/bin/llama-server

If you want a very detailed guide of making a very powerful llm-server build with TurboQuant KV-Caching, consider a this good walk-through. Even though it's a 'StudentLLM' - it is a full power-build that will run any model.

StudentLLM - Qwen2.5-coder-7b-instruct-q6-k / Qwen3.5 Agentic on a Ryzen 5-2600/ 3060ti. Production LLM or not? YES!

We Look a StudentLLM setup to get as much productivity out of limited hardware as we can.

Hot Configthinkmeltprotonmail.com

In our case our minimal run script required a fair bit of tweaking, and this is how we were able to get it to work.

• We were forced to make sure the mmproj file was not offloaded to GPU as this was breaking stuff, so our valid run configuration became:

/usr/bin/llama-server --jinja \
-m /home/c/models/Qwopus3.8-27B-Flash.i1-Q6_K.gguf \
--mmproj /home/c/models/mmproj-F32.gguf \
-ngl auto \
--no-mmproj-offload \
--host 192.168.1.3 \
--port 8080

One will notice we absolute pathed everything - it just works better out the gate. Instead of llama-server use /usr/bin/llama-server.  

Managing Expectations

This stuff is awesome, but it's not going to be instant or replace your professional subscription for Claude, or ChatGPT - nor should it with their budgets that rival small nations. The key is it's your LLM, and YOU now know how to build it. People in attorney-client privilege scenarios, doctors offices, privacy advocates simply cannot afford their data harvested.  Additionally not everybody wants to commit the $40 - $400/month to serious commerical grade LLMs. Additionally the investment can be somewhat modest as this runs on about $2000 in hardware.  That is where this site specializes in - small budgets, maximum performance on mini-bucks. You the small-guy are the winner!

Once it finally loads - your console back-end should look something like this:

First Run

We are not going to go too deeply into a standard LLM review as we have covered that a dozen times before, but will focus on the image-to-text aspects.

• This LLM is slow on our 4080 it came in at 4.8 T/s - ouch. Atypically it's been our experience that anything below 15 T/s is more of a testing LLM - and something people do not have the patience to wait on.

Image Recognition was Powerful and Fast!

• This LLM generates  slow - but we literally screen shot a log fired the image back at it, and it nailed it immediately, wow!
• It must be understood that Image handling LLM's will use the mmproj layers to convert the image into vector space of tokens, that is then effectively passed to a second LLM - the main one to discuss what is asked about the picture.

It took it only a few seconds to cleanly identify a cat

Can It Watch For Burglars - Absolutely!

It is pretty smart, really.

Conclusion

This LLM could have many, many functions. The sky is the limit. Monitor your 3D printer to make sure that your print is good, monitoring a process to make sure that it is operating safely. Could it be safety certified we think not - but it definitely can be a ongoing security scanner, and the cost is free - that's powerful and you are the benefactor.

In Part 1 we looked in detail at the X-11 calls in relation to graphics.  We simply wanted full control back of our widgets to make a button or a interface any way we wanted, and thus it spawned this project.  In Part 2 continued where we were able to make a couple very basic but completely working 'buttons'.   We then wanted to make them stylish thus in Part 3 we made a style manager that could track styles by name. Now our goal in Part 4 is just a simple polished button like you would see in Lazarus pascal,

Tbutton mybutton;

Because X-11 is basic it cannot actually draw a rounded button out of the gate - so we will need to construct one, a bit of planning suggests the following:

• Rectangle a, and b, will simply be draw overlaps, followed by 4 arc draws c, d, e, f.
• 4 line draw will make the outside, with 4 arc-lines to make the boundary.

Typically when you are mangling these it is illustrative to code  it as simply as you can, allowing for ample step debugging, in other words - code clearly and concisely you can always make your mysterious Gordian knot code later.

        XSetForeground(display_, gc_, style_.fillColor_);

        //region Region Rectangle A
        int a_x1 = x1_ + style_.radius_;
        int a_width = x2_ - x1_ -  2 * style_.radius_;
        int a_y1 = y1_;
        int a_height = y2_ - y1_;
        XFillRectangle(display_, window_, gc_, a_x1, a_y1, a_width, a_height);
        XFlush(display_);
        // endregion

        //region Region Rectangle B
        int b_x1 = x1_;
        int b_width = x2_ - x1_;
        int b_y1 = y1_ + style_.radius_;
        int b_height = y2_ - y1_  - 2 * style_.radius_;
        XFillRectangle(display_, window_, gc_, b_x1, b_y1, b_width, b_height);
        //endregion

This gives us when we call the draw function the following, which we have deliberately over sized it - so we can see what it looks like.

Once we are happy we can fold the variables back into a single function call as:

        //Rectangle A
        XFillRectangle(display_, window_, gc_, x1_ + style_.radius_, y1_, x2_ - x1_ -  2 * style_.radius_, y2_ - y1_);
        //Rectangle B
        XFillRectangle(display_, window_, gc_, x1_, y1_ + style_.radius_, x2_ - x1_,  y2_ - y1_  - 2 * style_.radius_);

We repeat this process and the entire fill with the 4 arcs looks like this :

        XSetForeground(display_, gc_, style_.fillColor_);

        //Rectangle A
        XFillRectangle(display_, window_, gc_, x1_ + style_.radius_, y1_, x2_ - x1_ -  2 * style_.radius_, y2_ - y1_);
        //Rectangle B
        XFillRectangle(display_, window_, gc_, x1_, y1_ + style_.radius_, x2_ - x1_,  y2_ - y1_  - 2 * style_.radius_);
        //region Arc C Top Left
        XFillArc(display_, window_, gc_, x1_, y1_, 2 * style_.radius_, 2 * style_.radius_, 90 * 64, 90 * 64);
        //region Arc D Top Right
        XFillArc(display_, window_, gc_, x2_ - 2 * style_.radius_, y1_, 2 * style_.radius_, 2 * style_.radius_, 0, 90 * 64);
        //region Arc E Bottom Right
        XFillArc(display_, window_, gc_, x2_ - 2 * style_.radius_, y2_ - 2 * style_.radius_, 2 * style_.radius_, 2 * style_.radius_, 270 * 64, 90 * 64);
        //region Arc F Bottom Left
        XFillArc(display_, window_, gc_, x1_, y2_ - 2 * style_.radius_, 2 * style_.radius_, 2 * style_.radius_, 180 * 64, 90 * 64);

        XFlush(display_);

Now it is producing a nice outline:

Were almost there - now we just need to add in the 4 line-outline, like so:

We also need to set our line width, we could of done it inside the style class, but we can really do it here, so:

        //Set Line Color
        XSetForeground(display_, gc_, style_.lineColor_);
        unsigned long valuemask = GCLineWidth;
        XGCValues gc_values;
        gc_values.line_width = style_.lineWidth_;
        XChangeGC(display_, gc_, valuemask, &gc_values );

        // 4 Line Bars
        XDrawLine(display_, window_, gc_, x1_ + style_.radius_, y1_, x2_ - style_.radius_, y1_);
        XDrawLine(display_, window_, gc_, x2_, y1_ + style_.radius_, x2_, y2_ - style_.radius_);
        XDrawLine(display_, window_, gc_, x1_, y1_ + style_.radius_, x1_, y2_ - style_.radius_);
        XDrawLine(display_, window_, gc_, x1_ + style_.radius_, y2_, x2_ - style_.radius_, y2_);

Now we have out 4 outlines

Finally we code up the arcs, which is simply the outline fill arcs, repeating and changing XFillArc to XDrawArc so we get:

    // 4 Arcs
    XDrawArc(display_, window_, gc_, x1_, y1_, 2 * style_.radius_, 2 * style_.radius_, 90 * 64, 90 * 64);
    //region Arc D Top Right
    XDrawArc(display_, window_, gc_, x2_ - 2 * style_.radius_, y1_, 2 * style_.radius_, 2 * style_.radius_, 0, 90 * 64);
    //region Arc E Bottom Right
    XDrawArc(display_, window_, gc_, x2_ - 2 * style_.radius_, y2_ - 2 * style_.radius_, 2 * style_.radius_, 2 * style_.radius_, 270 * 64, 90 * 64);
    //region Arc F Bottom Left
    XDrawArc(display_, window_, gc_, x1_, y2_ - 2 * style_.radius_, 2 * style_.radius_, 2 * style_.radius_, 180 * 64, 90 * 64);

Voila! Finally the last step is to write the text inside the button. This can be tricky because we need to know the extents of the font. So:

One More Step Writing the OK or Button Text

Finally we want to look at writing the Text Fill Button. Thusly this block should suffice:

        // Draw centered button text using the active style
        if (style_.font_)
        {
            XSetFont(display_, gc_, style_.font_->fid);
            XSetForeground(display_, gc_, style_.text_color_);

            int text_width = XTextWidth(style_.font_, text_.c_str(), static_cast<int>(text_.length()));
            int text_x = x1_ + ((x2_ - x1_) - text_width) / 2;
            int text_y = y1_ + ((y2_ - y1_) + style_.font_->ascent - style_.font_->descent) / 2;

            XDrawString(display_, window_, gc_, text_x, text_y, text_.c_str(), static_cast<int>(text_.length()));
        }

On Pause ATM

• We will continue this in a short bit, however a PILE of new models are in town so we are back to reviewing them!

This series builds on Part 1 - where we reviewed in detail how the X-11 system works, Part 2 where we made some buttons, with their own event handler system. Our buttons were basic, but they worked, and you now have full control over their size, no more wrangling with a layout manager deciding what's best for you! Why go to such extents to simple regain full control of your layouts?! - It's simple, suppose you wanted to make a instrumentation cluster for engineering, or a control interface for your raspberry pi? In the end this is worth doing!

But why not have nice buttons - stuff that looks like this? Well first we need to define styles...

Before you can do any coding in this you will need the support library as in:

sudo apt install libx11-dev -y

And your C++ source code will typically need to reference the header as in:

#include <X11/Xlib.h>
#include <X11/Xutil.h>

If you are using a CMakeLists.txt here is a good example that will work with X-11

cmake_minimum_required(VERSION 3.20)
project(XFrame LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
find_package(X11 REQUIRED)
add_executable(XFrame main.cpp)
target_link_libraries(XFrame PRIVATE X11::X11)

Our original button structure was very simple as in:

struct SButton
{
    std::string name;
    int x1, y1, x2, y2;
};

Now we will want to effectively add our own style class, and since this button manager really manages a set of buttons we can simple just have a std::vector of <SStyles> which will apply to the button set.   Typically one style of buttons will be active for a session. Naturally this is extensible to other widgets, and really generic, so in reality a good plan - is to just make it - it's own tracking class.  Once we move onto more advanced and different types of widgets our style manager will be independently ready, so we are putting in the work now for it.

Hence we are looking at an basic starting information structure of the following:

• Fill Color unsigned long - As set by X-11 (Internally it will be a Xcolor)
• Outline Color unsigned long  - As set by X-11 (Internally it will be a XColor)
• Corner Radius int  (X-11 does not draw with float precision unlike Qt
• Line Width  int
• Font Style XFontStruct

So from this it's a good start:

struct SStyle
{
    unsigned long fillColor_;
    unsigned long outlineColor_;
    unsigned long radius_;
    unsigned long lineWidth_;
    XFontStruct *font_;
};

Naturally we will need an vector array of these, and on top of this we will need a method to CRUD or create(add)  replace update and delete them, and really it would be nice for these setting to be able to save and load themselves at run-time. So really we will need a good class example.  However because we are using this as a demonstrator, and attempting to keep the tasks incrementally simple - we will not add object serialization at this time - that is where compound objects are either written to a binary format or to a json string.

Finally it can become problematic to have a index based list of styles without names so we update our structure as:

struct SStyle
{
  unsigned long fillColor_;
  unsigned long outlineColor_;
  unsigned long radius_;
  unsigned long lineWidth_;
  XFontStruct *font_;
  std::string style_name_;
};

struct Srgb
{
  int r;
  int g;
  int b;
};

• Srgb is to allow for passing a r, g, b value as one set.

A Very Basic Style Manager

class SStyle_Manager
{
private:
  std::vector<SStyle> styles_;
  Display *display_;
  Window window_;
  int screen_;
  GC gc_;
  size_t active_style = 0; // just set the default to first passed.
public:
  SStyle_Manager(Display *display, Window window, int screen, GC gc)
  {
    display_ = display;
    window_ = window;
    screen_ = screen;
    gc_ = gc;
  }
  ~SStyle_Manager()
  {
    for (auto& style : styles_) 
    {
      if (style.font_) 
      {
        XFreeFont(display_, style.font_);  // Free the X11 font resource
      }
    }
    styles_.clear(); 
  }

  int add_style(std::string styleName, Srgb fillColor, Srgb outlineColor, int radius, int linewidth,  std::string fontName)
  {
    SStyle nstyle;
    nstyle.style_name_ = std::move(styleName);
    nstyle.fillColor_ = get_color_Srgb(fillColor);
    nstyle.outlineColor_ = get_color_Srgb(outlineColor);
    nstyle.radius_ = radius;
    nstyle.lineWidth_ = linewidth;
    nstyle.font_ = XLoadQueryFont(display_, fontName.c_str());
    // Define fallback fonts to try if the requested font fails
    const char* fallbackFonts[] = { "fixed", "helvetica" };
    size_t numFallbacks = sizeof(fallbackFonts) / sizeof(*fallbackFonts);

    nstyle.font_ = nullptr;

    // Attempt to load the primary font (if provided and not empty)
    if (!fontName.empty()) {
      nstyle.font_ = XLoadQueryFont(display_, fontName.c_str());
    }

    // If the primary failed, iterate through fallback fonts until one succeeds or all are exhausted
    if (!nstyle.font_) {
      for (size_t i = 0; i < numFallbacks && !nstyle.font_; ++i) {
        nstyle.font_ = XLoadQueryFont(display_, fallbackFonts[i]);
      }
    }

    // Final validation: if no font loaded, report which fonts were attempted and exit
    if (!nstyle.font_)
    {
      printf("Failed to load any font (tried: ");
      if (!fontName.empty()) printf("%s", fontName.c_str());
      for (size_t i = 0; i < numFallbacks; ++i) {
        if (!fontName.empty() || i > 0) printf(", ");
        printf("%s", fallbackFonts[i]);
      }
      printf(")\n");
      return -1;
    }
    styles_.push_back(nstyle);
    return 0;
  }
  unsigned long get_color_Srgb(Srgb color)
  {
    XColor xcolor;
    xcolor.red = color.r << 8;
    xcolor.green = color.g << 8;
    xcolor.blue = color.b << 8;
    if (XAllocColor(display_, DefaultColormap(display_, screen_), &xcolor))
    {
      return xcolor.pixel;
    } else {
      printf("Failed to allocate color! RGB(%d,%d,%d)\n", color.r, color.g, color.b);
      return BlackPixel(display_, screen_);  // Fallback
    }
  }
  unsigned long get_color_rgb(unsigned char r, unsigned char g, unsigned char b)
  {
    XColor xcolor;
    xcolor.red = r << 8;
    xcolor.green = g << 8;
    xcolor.blue = b << 8;
    if (XAllocColor(display_, DefaultColormap(display_, screen_), &xcolor))
    {
      return xcolor.pixel;
    } else {
      printf("Failed to allocate color! RGB(%d,%d,%d)\n", r, g, b);
      return BlackPixel(display_, screen_);  // Fallback
    }
  }

};

We can go over all the parts, it will need some more functions to be utility ready, but anyways.

• std::vector<SStyle> styles_; will hold an array of different styles, enabling different theme layouts in a piece of software.
• Constructor takes Display *display_; Window window_; int screen_; GC gc_; and finally int active_style; will track any system hooks, this is needed for calls like XAllocColor which requires a hook into display_ so we simply pass it at constructor time. When it is done it looks like:

  SStyle_Manager(Display *display, Window window, int screen, GC gc)
  {
    display_ = display;
    window_ = window;
    screen_ = screen;
    gc_ = gc;
  }

• Destructor will make sure to free up the XFreeFont and it will look as:

  ~SStyle_Manager()
  {
    for (auto& style : styles_)
    {
      if (style.font_)
      {
        XFreeFont(display_, style.font_);  // Free the X11 font resource
      }
    }
    styles_.clear();
  }

• Color Converter get_color_Srgb will look as follows:

  unsigned long get_color_Srgb(Srgb color)
  {
    XColor xcolor;
    xcolor.red = color.r << 8;
    xcolor.green = color.g << 8;
    xcolor.blue = color.b << 8;
    if (XAllocColor(display_, DefaultColormap(display_, screen_), &xcolor))
    {
      return xcolor.pixel;
    } else {
      printf("Failed to allocate color! RGB(%d,%d,%d)\n", color.r, color.g, color.b);
      return BlackPixel(display_, screen_);  // Fallback
    }
  }

Finally the major add_style class which importantly will take a set of parameters, and will build a style object and stack it. It is a little extensive because we have coded in some font forgiveness which is just apropos, as we want it to default back to a system font if the passed font is not installed.

  int add_style(std::string styleName, Srgb fillColor, Srgb outlineColor, int radius, int linewidth,  std::string fontName)
  {
    SStyle nstyle;
    nstyle.style_name_ = std::move(styleName);
    nstyle.fillColor_ = get_color_Srgb(fillColor);
    nstyle.outlineColor_ = get_color_Srgb(outlineColor);
    nstyle.radius_ = radius;
    nstyle.lineWidth_ = linewidth;
    nstyle.font_ = XLoadQueryFont(display_, fontName.c_str());
    // Define fallback fonts to try if the requested font fails
    const char* fallbackFonts[] = { "fixed", "helvetica" };
    size_t numFallbacks = sizeof(fallbackFonts) / sizeof(*fallbackFonts);

    nstyle.font_ = nullptr;

    // Attempt to load the primary font (if provided and not empty)
    if (!fontName.empty()) {
      nstyle.font_ = XLoadQueryFont(display_, fontName.c_str());
    }

    // If the primary failed, iterate through fallback fonts until one succeeds or all are exhausted
    if (!nstyle.font_) {
      for (size_t i = 0; i < numFallbacks && !nstyle.font_; ++i) {
        nstyle.font_ = XLoadQueryFont(display_, fallbackFonts[i]);
      }
    }

    // Final validation: if no font loaded, report which fonts were attempted and exit
    if (!nstyle.font_)
    {
      printf("Failed to load any font (tried: ");
      if (!fontName.empty()) printf("%s", fontName.c_str());
      for (size_t i = 0; i < numFallbacks; ++i) {
        if (!fontName.empty() || i > 0) printf(", ");
        printf("%s", fallbackFonts[i]);
      }
      printf(")\n");
      return -1;
    }
    styles_.push_back(nstyle);
    return 0;
  }

This is pretty close to complete - what is missing is get_active_style and set_active_style and since we have built our style tracker with a super nice style_name_ we get to use that!  This gives real flexibility to the developer because they can simply go set_active_style_byName('default') and if there is a registered style with that name it will automatically become the default.

Adding a set_active_style_byName then becomes:

  int set_active_style_byName(std::string styleName)
  {
    auto it = std::find_if(styles_.begin(), styles_.end(),
        [&](const SStyle& s) { return s.style_name_ == styleName; });

    if (it != styles_.end())
    {
      active_style = static_cast<int>(std::distance(styles_.begin(), it));
      return active_style;
    }
    return -1; // Not found
  }

And some get_active_style  functions thusly:

  SStyle get_active_style_byName(std::string styleName)
  {
    auto it = std::find_if(styles_.begin(), styles_.end(),
      [&](const SStyle& s) { return s.style_name_ == styleName; });
  }
  SStyle get_active_style()
  {
    return styles_[active_style];
  }
  SStyle get_index_style(int index)
  {
    if (index > 0 && index < (int)styles_.size())
    {
      return styles_[index];
    }
  }

The entire code block now looks as (I know there are some overloaded #includes but anyhow..)

//
// Created by c on 8/31/26.
//

#include <X11/Xlib.h>
#include <X11/Xutil.h>
#include <X11/fonts/font.h>
#include <algorithm>
#include <cstdint>
#include <cstdlib>
#include <utility>
#include <vector>
#include <algorithm>
#include <cstring>
#include <iostream>
#include <string>
#include <stdio.h>
#include <memory>


struct SStyle
{
  unsigned long fillColor_;
  unsigned long outlineColor_;
  unsigned long radius_;
  unsigned long lineWidth_;
  XFontStruct *font_;
  std::string style_name_;
};

struct Srgb
{
  int r;
  int g;
  int b;
};

class SStyle_Manager
{
private:
  std::vector<SStyle> styles_;
  Display *display_;
  Window window_;
  int screen_;
  GC gc_;
  int active_style = 0; // just set the default to first passed.
public:
  SStyle_Manager(Display *display, Window window, int screen, GC gc)
  {
    display_ = display;
    window_ = window;
    screen_ = screen;
    gc_ = gc;
  }
  ~SStyle_Manager()
  {
    for (auto& style : styles_)
    {
      if (style.font_)
      {
        XFreeFont(display_, style.font_);  // Free the X11 font resource
      }
    }
    styles_.clear();
  }
  int add_style(std::string styleName, Srgb fillColor, Srgb outlineColor, int radius, int linewidth,  std::string fontName)
  {
    SStyle nstyle;
    nstyle.style_name_ = std::move(styleName);
    nstyle.fillColor_ = get_color_Srgb(fillColor);
    nstyle.outlineColor_ = get_color_Srgb(outlineColor);
    nstyle.radius_ = radius;
    nstyle.lineWidth_ = linewidth;
    nstyle.font_ = XLoadQueryFont(display_, fontName.c_str());
    // Define fallback fonts to try if the requested font fails
    const char* fallbackFonts[] = { "fixed", "helvetica" };
    size_t numFallbacks = sizeof(fallbackFonts) / sizeof(*fallbackFonts);

    nstyle.font_ = nullptr;

    // Attempt to load the primary font (if provided and not empty)
    if (!fontName.empty()) {
      nstyle.font_ = XLoadQueryFont(display_, fontName.c_str());
    }

    // If the primary failed, iterate through fallback fonts until one succeeds or all are exhausted
    if (!nstyle.font_) {
      for (size_t i = 0; i < numFallbacks && !nstyle.font_; ++i) {
        nstyle.font_ = XLoadQueryFont(display_, fallbackFonts[i]);
      }
    }

    // Final validation: if no font loaded, report which fonts were attempted and exit
    if (!nstyle.font_)
    {
      printf("Failed to load any font (tried: ");
      if (!fontName.empty()) printf("%s", fontName.c_str());
      for (size_t i = 0; i < numFallbacks; ++i) {
        if (!fontName.empty() || i > 0) printf(", ");
        printf("%s", fallbackFonts[i]);
      }
      printf(")\n");
      return -1;
    }
    styles_.push_back(nstyle);
    return 0;
  }
  unsigned long get_color_Srgb(Srgb color)
  {
    XColor xcolor;
    xcolor.red = color.r << 8;
    xcolor.green = color.g << 8;
    xcolor.blue = color.b << 8;
    if (XAllocColor(display_, DefaultColormap(display_, screen_), &xcolor))
    {
      return xcolor.pixel;
    } else {
      printf("Failed to allocate color! RGB(%d,%d,%d)\n", color.r, color.g, color.b);
      return BlackPixel(display_, screen_);  // Fallback
    }
  }
  unsigned long get_color_rgb(unsigned char r, unsigned char g, unsigned char b)
  {
    XColor xcolor;
    xcolor.red = r << 8;
    xcolor.green = g << 8;
    xcolor.blue = b << 8;
    if (XAllocColor(display_, DefaultColormap(display_, screen_), &xcolor))
    {
      return xcolor.pixel;
    } else {
      printf("Failed to allocate color! RGB(%d,%d,%d)\n", r, g, b);
      return BlackPixel(display_, screen_);  // Fallback
    }
  }
  int set_active_style_byName(std::string styleName)
  {
    auto it = std::find_if(styles_.begin(), styles_.end(),
        [&](const SStyle& s) { return s.style_name_ == styleName; });

    if (it != styles_.end())
    {
      active_style = static_cast<int>(std::distance(styles_.begin(), it));
      return active_style;
    }
    return -1; // Not found
  }
  SStyle get_active_style_byName(std::string styleName)
  {
    auto it = std::find_if(styles_.begin(), styles_.end(),
      [&](const SStyle& s) { return s.style_name_ == styleName; });
  }
  SStyle get_active_style()
  {
    return styles_[active_style];
  }
  SStyle get_index_style(int index)
  {
    if (index > 0 && index < (int)styles_.size())
    {
      return styles_[index];
    }
  }

};

Conclusion

We are goingto stop there for today, we have a full basic style manager that will allow our future application to have various themes that we can allow the user to select, yes it's very basic, but you can see that it is very extensible.  In our next article we will look at integrating the style manager and making buttons a standalone class, not a button manager. The idea behind this is a small application does not need a giant widget tracker and can make one class instance for the widgets that it desires!  Notice no we have not coded up a delete function - there is a challenge for you to figure out!

In our previous post we studied the X-11 Framework in detail,

Writing Your Own X-11 Framework Basics (Part 1)

We go in depth into a X-11 foundational primer.

Hot Configthinkmeltprotonmail.com

We learned the very basics of a display which is either a virtual reference or a real reference which differs from the screen which is your monitors 0-2 in a three-header monitor setup.  We also learned that the graphical context is the 'pen' that you use to write with. We look at setting masks so that your application will receive various events like a mouse click or keypress.   Finally  we learned the various drawing commands such as XDrawLines and XDrawRectangles which we can use to make primitive drawings, and writing text with XDrawString

But that is where it ends, we have to build our buttons, labels and editbox from scratch, but that's the fun part - we can now create any type of widget we want.

To make this very simple we will start with virtual button manager  that will print to the console when the user clicks inside the corresponding button window, so a simple example:

Before any X-11 development can be done, you will need the support libraries:

sudo apt install libx11-dev -y

Also you typically will need the following if you are using CMakeLists.txt

cmake_minimum_required(VERSION 3.20)
project(XFrame LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
find_package(X11 REQUIRED)
add_executable(XFrame main.cpp)
target_link_libraries(XFrame PRIVATE X11::X11)

When this is done the buttons will look like this:

#include <X11/Xlib.h>
#include <X11/Xutil.h>
#include <X11/fonts/font.h>
#include <algorithm>
#include <cstdint>
#include <cstdlib>
#include <vector>
#include <algorithm>
#include <cstring>
#include <iostream>
#include <string>
#include <stdio.h>
#include <memory>


struct SButton
{
    std::string name;
    int x1, y1, x2, y2;
};
//region Button_Manager
class Button_Manager
{
private:
    std::vector<SButton> buttons;
    Display *display_;
    Window window_;
    int screen_;
    GC gc_;

    unsigned long grey_color_ = 0;
    unsigned long bright_grey_ = 0;
    unsigned long bright_red_ = 0;

public:
    Button_Manager(Display* display, int screen, Window window, GC gc)
    {
        display_ = display;
        screen_ = screen;
        window_ = window;
        gc_ = gc;
        grey_color_ = get_color_rgb(25, 25, 25);
        bright_grey_ = get_color_rgb( 55, 55, 55);
        bright_red_ = get_color_rgb(205, 35, 35);
    }
    ~Button_Manager(){}
    void add_button(int x1, int y1, int x2, int y2, std::string name)
    {
        SButton button;
        button.x1 = x1;
        button.y1 = y1;
        button.x2 = x2;
        button.y2 = y2;
        button.name = name;
        buttons.push_back(button);
    }
    void add_buttonwh(int x1, int y1, int width, int height, std::string name)
    {
        SButton button;
        button.x1 = x1;
        button.y1 = y1;
        button.x2 = x1 + width;
        button.y2 = y1 + height;
        button.name = name;
        buttons.push_back(button);
    }
    std::string clicked(int x, int y)
    {
        for (size_t i = 0; i < buttons.size(); i++)
        {
            if ((x >= buttons[i].x1 && x <= buttons[i].x2) && (y >= buttons[i].y1 && y <= buttons[i].y2))
            {
            return buttons[i].name;
            }
        }
    }
    unsigned long get_color_rgb(unsigned char r, unsigned char g, unsigned char b)
    {
        XColor xcolor;
        xcolor.red = r << 8;
        xcolor.green = g << 8;
        xcolor.blue = b << 8;
        if (XAllocColor(display_, DefaultColormap(display_, screen_), &xcolor))
        {
            return xcolor.pixel;
        } else {
            printf("Failed to allocate color! RGB(%d,%d,%d)\n", r, g, b);
            return BlackPixel(display_, screen_);  // Fallback
        }
    }
    void center_text_in_button(SButton button)
    {

        std::string ref_text = button.name;

        // Load and query the font (using "fixed" as a common default)
        XFontStruct *font = XLoadQueryFont(display_, "-misc-fixed-medium-r-normal--9-*");

        if (!font)
        {
            printf("Failed to load font\n");
            return;
        }

        int x = button.x1 + 3;
        int y = button.y2 - 3;


        XSetForeground(display_, gc_, bright_red_);
        XDrawString(display_, window_, gc_, x, y, ref_text.c_str(), strlen(ref_text.c_str()));

        // Clean up when done with font operations
        XFreeFont(display_, font);
    }

    void draw_buttons()
    {
        for (size_t i = 0; i < buttons.size(); ++i)
        {
            SButton button = buttons[i];
            int width = button.x2 - button.x1;
            int height = button.y2 - button.y1;
            XSetForeground(display_, gc_, grey_color_);
            XFillRectangle(display_, window_, gc_,  button.x1, button.y1, width, height);
            XSetForeground(display_, gc_, bright_grey_);
            XDrawRectangle(display_, window_, gc_, button.x1, button.y1, width, height);
            center_text_in_button(button);
        }
        XFlush(display_);
    }
};


int main()
{
    Display* display = XOpenDisplay(nullptr);
    if (!display) {return 1; }
    int screen = DefaultScreen(display);
    XID Xid = RootWindow(display, screen);
    Window window = XCreateSimpleWindow(display, Xid,100,100,800,600,1, WhitePixel(display, screen), BlackPixel(display, screen));
    XStoreName(display, window, "My C++ Canvas Framework");
    XSelectInput(display, window, ButtonPressMask | KeyPressMask | ExposureMask);
    Atom deleteWindowMessage = XInternAtom(display, "WM_DELETE_WINDOW", False);
    XSetWMProtocols(display,window,&deleteWindowMessage,1);
    XMapWindow(display, window);
    GC gc = DefaultGC(display, screen);
    Button_Manager bm(display, screen, window, gc);
    bm.add_buttonwh(10, 10, 80, 20, "Test Button a");
    bm.add_buttonwh(10,50, 80, 20, "Test Button b");
    bm.draw_buttons();
    bool running = true;
    while (running)
    {
        XEvent event;
        if (!XPending(display)) {continue;}
        XNextEvent(display, &event);
        switch (event.type)
        {
            case ButtonPress:
            {
                int x1 = event.xbutton.x, y1 = event.xbutton.y;
                std::string name = bm.clicked(event.xbutton.x, event.xbutton.y);
                std::cout << "You pressed:" << name << std::endl;
            }
            case Expose:
            {
                bm.draw_buttons();
                break;
            }
            case ConfigureNotify:
            {
                break;
            }
            case KeyPress:
            {
                break;
            }
            case ClientMessage:
            {
                break;
            }
        }
    }
    XCloseDisplay(display);
    return 0;
}

Going over this code in detail, starting with main:

int main()
{
    Display* display = XOpenDisplay(nullptr);
    if (!display) {return 1; }
    int screen = DefaultScreen(display);
    XID Xid = RootWindow(display, screen);
    Window window = XCreateSimpleWindow(display, Xid,100,100,800,600,1, WhitePixel(display, screen), BlackPixel(display, screen));
    XStoreName(display, window, "My C++ Canvas Framework");
    XSelectInput(display, window, ButtonPressMask | KeyPressMask | ExposureMask);
    Atom deleteWindowMessage = XInternAtom(display, "WM_DELETE_WINDOW", False);
    XSetWMProtocols(display,window,&deleteWindowMessage,1);
    XMapWindow(display, window);
    GC gc = DefaultGC(display, screen);
    Button_Manager bm(display, screen, window, gc);
    bm.add_buttonwh(10, 10, 80, 20, "Test Button a");
    bm.add_buttonwh(10,50, 80, 20, "Test Button b");

• We ask for a Display* display which is the virtual or real result from a call to XOpenDisplay(nullptr);  Recall that in linux this can be a completely virtual desktop.. If (!display) kick back an error and exit.
• We then get the corresponding screen (typicallly 0) with int screen = DefaultScreen(display)  Do recall if you have multiple-head monitors and launch your application into screen 1, 2 it will return that.
• Again XID Xid = RootWindow(display, screen); is simply a UID resource of the display screen combination when we called it we receieved 836, however it is needed so that we can create the window in the next line
• Window window = XCreateSimpleWindow(display, Xid,100,100,800,600,1, WhitePixel(display, screen), BlackPixel(display, screen)); sets the dimensions for our application, starting at a x,y offset of 100, 100 and with a width: 800 and height 600
• XStoreName sets the title for the application.
• XSelectInput(display, window, ButtonPressMask | KeyPressMask | ExposureMask) sets up our event handlers.  For a full list of them see the base article that this is based off of:

Writing Your Own X-11 Framework Basics (Part 1)

We go in depth into a X-11 foundational primer.

Hot Configthinkmeltprotonmail.com

• The next lines set up default destructor handling, sets a default context etc;

 Atom deleteWindowMessage = XInternAtom(display, "WM_DELETE_WINDOW", False);
    XSetWMProtocols(display,window,&deleteWindowMessage,1);
    XMapWindow(display, window);
    GC gc = DefaultGC(display, screen);

Custom Button Handler

This is where the fun begins, we have written our own custom basic button handler class. It needs several working parts.

• A Constructor that logs the display, screen, window, and gc information at creation time.  By doing it this way the rest of the calls no longer need to record it, and it is dutifully recorded in the constructor call:

    Button_Manager(Display* display, int screen, Window window, GC gc)
    {
        display_ = display;
        screen_ = screen;
        window_ = window;
        gc_ = gc;
        grey_color_ = get_color_rgb(25, 25, 25);
        bright_grey_ = get_color_rgb( 55, 55, 55);
        bright_red_ = get_color_rgb(205, 35, 35);
    }

• Of importance we pre-register the colors that we want to use with the get_color_rgb function which is listed in the code.

Next we have a add_button that will take it's x1, y1, x2, y2, name as parameters or alternately you can add it as x1, y1, width, height, name with the add_buttonwh function.

Event Handler

• The event handler is quite simple, any mouse clicks are passed to the class handler and it will check the x,y dimensions if they are inside the corresponding box, if they are it will return the std::string of the name of the button that was clicked.

    std::string clicked(int x, int y)
    {
        for (size_t i = 0; i < buttons.size(); i++)
        {
            if ((x >= buttons[i].x1 && x <= buttons[i].x2) && (y >= buttons[i].y1 && y <= buttons[i].y2))
            {
            return buttons[i].name;
            }
        }
    }

Draw Buttons

Finally we have a draw buttons, and hence the requirement that the class object record all the pertinent information from the system so that a call to draw buttons is simply:

bm.drawbuttons();

The draw buttons will look as:

    void draw_buttons()
    {
        for (size_t i = 0; i < buttons.size(); ++i)
        {
            SButton button = buttons[i];
            int width = button.x2 - button.x1;
            int height = button.y2 - button.y1;
            XSetForeground(display_, gc_, grey_color_);
            XFillRectangle(display_, window_, gc_,  button.x1, button.y1, width, height);
            XSetForeground(display_, gc_, bright_grey_);
            XDrawRectangle(display_, window_, gc_, button.x1, button.y1, width, height);
            center_text_in_button(button);
        }
        XFlush(display_);
    }

center_text_in_button will use the red to draw the text.

XFlush(display_) is required as all the XFillRectangle and XDrawRectangle will not actually execute until XFlush is required.

Conclusion

• This is a basic button drawing application but we are no longer fighting with another framework manager's decision to force oversize our buttons.  Yes our layouts are pretty basic, but now we can create any custom button / widget that we desire!  So naturally the next is to look at some goals
• Making stylish buttons
• Making edit / text boxes that can accept input
• Making more advanced widgets!

Finally at the end make a drawing IDE that will let you simply draw an application that will spit out fast boiler plate code!

Many many moons ago a wonderful piece of software came out for Window 95, namely Delphi 2.0.  It was amazing, and  let you literally draw your buttons and applications, connect them to event handlers, and then insert some boilerplate Turbo Pascal code.  Today you would realize this was 25 years ahead of it's time, and basically long buried and gone after many attempts by Windows to bury it as it competed with their myriad of various IDE produtcs. It survived to a basic fashion, and  you can still pay huge dollars and buy Embarcadero products that have bought and traded hands many times, or the free Lazarus IDE, which was an opensource clone of the powerful Delphi products.

But as for a C / C++ based type GUI (in Linux) if you are not going the Embarcadero route,  you are pretty much stuck mostly with Qt.  Qt is really nice, however if you want total control like say a custom sized button that acts like a instrument you end up fighting the layout manager which will decide for you that your button needs to be a minimum of X pixels, or the font needs to be N size. Then the restrictive licensing agreements will helm you in.  In the end you waste days getting a generic 'toy' layout you never wanted.

So what if we just wrote  one from scratch! Secondly why even write this article when the AI's do it so well..  The answer is really simple to really learn something you need to spend some time in it, not just glancing through code but documenting each part one bit at a time.  It could be noted that 'vibe-coding' something to come back to it even a week later and you have no understanding or recollection of how it works, or what it does is just powerless - and dangerous.  AI's will get 98% of stuff right, but the 2% that they get wrong will trap you.  If you don't have a good understanding and can work close to the level that the AI is working at you will be trapped. That is the goal of the series of these articles to slowly build a strong foundational knowledge of the X-11 X-Window System, and eventually build an entire RAD (Rapid Application Developer) or IDE.

Supports:

Before you can write any X-11 application make sure to reference it in your source, and to also install the support library, so:

sudo apt install libx11-dev -y

#include <X11/Xlib.h>
#include <X11/Xutil.h>

Here additionally is a working CMakeLists.txt if it helps your project.

cmake_minimum_required(VERSION 3.20)
project(XFrame LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
find_package(X11 REQUIRED)
add_executable(XFrame main.cpp)
target_link_libraries(XFrame PRIVATE X11::X11)

BoilerPlate Introduction

int main()
{
    Display* display = XOpenDisplay(nullptr);
    if (!display) {return 1; }
    int screen = DefaultScreen(display);
    XID Xid = RootWindow(display, screen);
    Window window = XCreateSimpleWindow(display, Xid,100,100,800,600,1, BlackPixel(display, screen), WhitePixel(display, screen));
    XStoreName(display, window, "My C++ Canvas Framework");
    XSelectInput(display, window, ExposureMask | KeyPressMask | StructureNotifyMask );
    Atom deleteWindowMessage = XInternAtom(display, "WM_DELETE_WINDOW", False);
    XSetWMProtocols(display,window,&deleteWindowMessage,1);
    XMapWindow(display, window);
    GC gc = DefaultGC(display, screen);
    Canvas canvas(display, window, gc, 800, 600);
    bool running = true;
    while (running)
    {
        XEvent event;
        XNextEvent(display, &event);
        switch (event.type)
        {
        case Expose:
            canvas.present();
            break;

Going over this snippet of code let's just study the different parts:

• Display* display = XOpenDisplay(nullptr); This will get the current display.  Please note - this is not necessarily the screen as X-11 can have virtual displays, but will get the active one that the current session is calling to use.  So if you open an application inside your Linux Gui, this is the display.  if (!display) {return 1;} is simply saying if no display is returned exit with error code  1 versus 0 (no error code.)
• int screen = DefaultScreen(display); If you have a 3-head monitor setup this will get the default monitor of the group. Typically 0
• XID Xid = RootWindow(display, screen) Effectively this is simply the resource identifier that the curent display / sreen combination is using.  When we step-debugged our code we received Xid: 883
• Window window = XCreateSimpleWindow(display, Xid,100,100,800,600,1, BlackPixel(display, screen), WhitePixel(display, screen)); This actually creates a window, however it will not be displayed just yet. It exists virtually. display the current display, Xid - the resource identifier of the display, screen combination, 100,100 is x,y coordinates from top-left, 800,600 is the width, height.  1 is the border width, and BlackPixel(display,screen) border, and a WhitePixel(display,screen) background.
• XStoreName(display, window, "My C++ Canvas Framework"); Will give this (still virtual) window a name for it's title bar.

Event Signal Handling

• XSelectInput(display, window, ExposureMask | KeyPressMask | StructureNotifyMask ); will determine which events will be passed to the window for handling. An event is - did your mouse move? Does your window need to know about it, then we add a corresponding mask.

Here is a list of possible masks:

The standard event masks available to XSelectInput are:

Specifically  if we want button based event handlers it becomes:

The mouse and pointer-related masks available to XSelectInput are:

Typically a normal mouse interaction type mask setting would look like:

XSelectInput(display, window, ExposureMask | KeyPressMask |    StructureNotifyMask | ButtonPressMask | ButtonReleaseMask | PointerMotionMask | EnterWindowMask |  LeaveWindowMask
);

From this you would then have the following code block to handle these events, namely:

switch (event.type) {
case ButtonPress: {
    int x = event.xbutton.x;
    int y = event.xbutton.y;
    unsigned int button = event.xbutton.button;

    // Handle mouse press.
    break;
}

case ButtonRelease: {
    int x = event.xbutton.x;
    int y = event.xbutton.y;
    unsigned int button = event.xbutton.button;

    // Handle mouse release.
    break;
}

case MotionNotify: {
    int x = event.xmotion.x;
    int y = event.xmotion.y;

    // Handle pointer movement.
    break;
}

case EnterNotify:
    // Pointer entered the window.
    break;

case LeaveNotify:
    // Pointer left the window.
    break;
}

Finishing up this block of code we still have:

Atom deleteWindowMessage = XInternAtom(display, "WM_DELETE_WINDOW", False);
XSetWMProtocols(display,window,&deleteWindowMessage,1);
XMapWindow(display, window);
GC gc = DefaultGC(display, screen);
Canvas canvas(display, window, gc, 800, 600);

• Atom deleteWindowMessage = XInteralAtom(display, "WM_DELETE_WINDOW", False); This is telling the system to give us it's internal Atom reference of a WM_DELETE_WINDOW message as we want to intercept it and handle it manually.
• XSetWMProtocols(display, window, &deleteWindowMessage, 1);  Once we have the referenced Atom we then use that address to reference back that our application supports the WM_DELETE_WINDOW protocol.
• XMapWindow(display, window) will make your application visible.

Picking your Drawing Context (Picking your Pen to Draw With)

Finally - yep I know this is a LOT, but were almost there, we need to get first the drawing context,

• A drawing context is only settings - it describes the pen you are holding in your hand at the time you used it to write.  
• For instance you might have one drawing context that uses line width 5 and a second that uses red.  Again think of them as pens.  
• The default drawing context is obtained with :

GC gc = DefaultGC(display, screen);

Drawing Commands

XDrawPoint

XDrawPoint(display, window, gc, 100, 80);

• XDrawPoint Of the current display for the current window using the drawing context of gc create a single point at 100, 80. This command queues until xflush(display) is called.

XDrawPoints

XPoint points[] = {{100, 80}, {110, 90}, {120, 100}, {130, 110}};
XDrawPoints(display, window, gc,points, 4, CoordModeOrigin);

• XDrawPoints Of the current display for the current window using the drawing context of gc here are 4 coordinate points. CoordModeOrigin specifies that each point is absolute, while CoordModePrevious indicates the first point is absolute, and each point thereafter is relative. This command queues until xflush(display) is called.

XDrawLine

XDrawLine(display, window, gc,
    50, 150,       // starting x, y
    300, 200       // ending x, y
);

• XDrawLine of the current display for the current window using the drawing context of gc draw a line from 50,150 to 300,120 This command queues until xflush(display) is called.

XDrawLines

XPoint linePoints[] = {{50, 250}, {150, 200}, {250, 250}, {350, 200}};

XDrawLines(display, window, gc, linePoints, 4, CoordModeOrigin);

• XDrawLines Of the current display of the current window using the drawing context of gc here are 4 consecutive points that will will draw contiguously. This command queues until xflush(display) is called.

XDrawSegments

XSegment segments[] = {
    {50, 300, 150, 350},
    {200, 300, 300, 350},
    {350, 300, 450, 350}
};

XDrawSegments(display, window, gc, segments, 3);

• XDrawSegments Of the current display of the current window using the drawing context of gc  here are 3 line segments that can each be drawn. This command queues until xflush(display) is called.

XDrawRectangle

XDrawRectangle(display, window, gc, 
    50, 400,       // x, y
    150, 80        // width, height
);

• XDrawRectangle Of the current display of the current window using the drawing context of gc draw a rectangle starting at 50,400 with a width,height of 150,80 This command queues until xflush(display) is called.

XDrawRectangles

XRectangle outlinedRectangles[] = {
    {250, 400, 100, 60},
    {400, 400, 100, 80},
    {550, 400, 150, 50}
};

XDrawRectangles(display, window, gc, outlinedRectangles,3);

• XDrawRectangles Of the current display of the current window using the drawing context of gc draw the following 3 rectangles as passed in outlinedRectangles This command queues until xflush(display) is called.

XFillRectangle

XFillRectangle(
    display,
    window,
    gc,
    50, 500,       // x, y
    150, 60        // width, height
);

• XFillRectangle Of the current display of the current window using the drawing context of gc draw the following filled rectangle This command queues until xflush(display) is called.  
• Does not put a line around the rectangle, you must make a second call to XDrawRectangle

One may set the fill with:

XSetFillStyle(display, gc, FillSolid);
XSetForeground(display, gc, 0xFF0000);
// Four fill styles are offered: 
// FillSolid
// FillTiled
// FillStippled
// FillOpaqueStippled

XFillRectangles

XRectangle filledRectangles[] = {
    {250, 500, 100, 50},
    {400, 500, 100, 70},
    {550, 500, 150, 40}
};

XFillRectangles(display, window, gc, filledRectangles, 3);

• XFillRectangles Of the current display of the current window using the drawing context of gc draw the following set of filled rectangles This command queues until xflush(display) is called.

XDrawArc

• Angles are measured in arc seconds each second is 1/64th of a degree, thus to draw a 1/2 circle becomes 180 * 64

XDrawArc(
    display,
    window,
    gc,
    50, 50,             // bounding box x, y
    150, 100,           // bounding box width, height
    0,                  // starting angle
    180 * 64            // angle extent
);

• XDrawArc Of the current display of the current window using the drawing context of gc draw the following arc starting at reference point 50, 50 and bounded by 150,100 with a starting angle of 0 degrees. This command queues until xflush(display) is called.

XDrawArcs

• Angles are measured in arc seconds each second is 1/64th of a degree, thus to draw a 1/2 circle becomes 180 * 64

XArc outlinedArcs[] = {
    {250, 50, 100, 100, 0,       360 * 64}, // circle
    {400, 50, 100, 100, 0,       180 * 64}, // half-circle
    {550, 50, 150, 100, 90 * 64, 180 * 64}
};

XDrawArcs(display, window, gc,outlinedArcs, 3);

• XDrawArcs Of the current display of the current window using the drawing context  of gc draw the following 3 arcs each referenced by x, y, width, height, starting angle, angle extent  Thus each arc is described by 6 points of data. This command queues until xflush(display) is called.

XFillArc

• Angles are measured in arc seconds each second is 1/64th of a degree, thus to draw a 1/2 circle becomes 180 * 64

XFillArc(
    display,
    window,
    gc,
    50, 180,             // bounding box x, y
    150, 150,            // width, height
    0,                   // starting angle
    90 * 64              // angle extent
);

• XFillArc Of the current display of the current window using the drawing context  of gc draw and fill in the following arc referenced by x, y, width, height, starting angle, angle extent  Thus each arc is described by 6 points of data. This command queues until xflush(display) is called.

XFillArcs

XArc filledArcs[] = {
    {250, 180, 100, 100, 0,        360 * 64}, // filled circle
    {400, 180, 100, 100, 0,        180 * 64}, // filled half-circle
    {550, 180, 150, 100, 90 * 64,  180 * 64}
};

XFillArcs(
    display,
    window,
    gc,
    filledArcs,
    3
);

• XFillArcs Of the current display of the current window using the drawing context  of gc draw the following 3 arcs each referenced by x, y, width, height, starting angle, angle extent  Thus each arc is described by 6 points of data. This command queues until xflush(display) is called.

XDrawString

XDrawString(
    display,
    window,
    gc,
    50, 600,             // x and baseline y
    text,
    std::strlen(text)
);

• XDrawString Of the current display of the current window using the drawing context of gc at x,y point 50,600 write the following text that has a length of std::strlen(text). This command queues until xflush(display) is called.

XDrawImageString

XDrawImageString(
    display,
    window,
    gc,
    250, 600,
    text,
    std::strlen(text)
);

• XDrawImageString Of the current display of the current window using the drawing context of gc at x,y point 250,600 write the following text that has a length of std::strlen(text) and give it a rectangular background. This command queues until xflush(display) is called.

XDrawString16

XChar2b text16[] = {
    {0, 'H'},
    {0, 'i'},
    {0, '!'}
};

// Draw 16-bit text
XDrawString16(
    display,
    window,
    gc,
    400, 600,
    text16,
    3
);

• XDrawString16 Of the current display of the current window using the drawing context of gc at x,y point 50,600 write the following 16-bit text that has a length of 3. This command queues until xflush(display) is called.

XDrawText

XTextItem textItems[] = {
    {(char*)"Hello ", 6, 0, None},
    {(char*)"X11",    3, 0, None}
};

XDrawText(
    display,
    window,
    gc,
    50, 580,             // x and baseline y
    textItems,
    2                    // number of text items
);

• XDrawText Of the current display of the current window using the drawing context of gc at x,y points 50, 580 write out the following textItems of count 2. This command queues until xflush(display) is called.

Finishing the Frame.

Once you have buffered up all your XDraw commands you  can send them for processing with

XFlush(display);

Conclusion

This is a good primer. We can see in the documentation that the raw X-11 does not natively handle buttons etc.  Below is a entire working X-11 with a Canvas class to to various updates etc:

#include <X11/Xlib.h>
#include <X11/Xutil.h>

#include <algorithm>
#include <cstdint>
#include <cstdlib>
#include <vector>

//region Region Canvas
class Canvas
{
private:
    Display* display = nullptr;
    Window window = 0;
    GC gc = nullptr;

    XImage* image = nullptr;

    int width = 0;
    int height = 0;

    std::vector<uint32_t> pixels;


public:
    Canvas(Display* display, Window window, GC graphicsContext,int width, int height): display(display), window(window), gc(graphicsContext)
    {
        resize(width, height);
    }
    ~Canvas() { destroyImage(); }
    void resize(int newWidth, int newHeight)
    {
        width = std::max(1, newWidth);
        height = std::max(1, newHeight);
        pixels.resize(static_cast<size_t>(width) * height);
        clear(0x202020);
        destroyImage();
        Visual* visual = DefaultVisual(display, DefaultScreen(display));
        int depth = DefaultDepth(display, DefaultScreen(display));

        /*
         * XCreateImage calculates bytes_per_line for us when the
         * data pointer is initially null.
         */
        image = XCreateImage(display, visual, depth, ZPixmap, 0, nullptr, width, height, 32, 0);

        if (!image)
        {
            std::exit(1);
        }
        image->data = static_cast<char*>(std::malloc(image->bytes_per_line * height));
        if (!image->data)
        {
            std::exit(1);
        }
    }
    int getWidth() const
    {
        return width;
    }
    int getHeight() const
    {
        return height;
    }
    // Direct access to the raw framebuffer.
    uint32_t* data()
    {
        return pixels.data();
    }
    const uint32_t* data() const
    {
        return pixels.data();
    }
    void clear(uint32_t rgb)
    {
        std::fill(pixels.begin(), pixels.end(), rgb);
    }
    void setPixel(int x, int y, uint32_t rgb)
    {
        if (x < 0 || x >= width || y < 0 || y >= height)
        {
            return;
        }
        pixels[static_cast<size_t>(y) * width + x] = rgb;
    }
    uint32_t getPixel(int x, int y) const
    {
        if (x < 0 || x >= width || y < 0 || y >= height)
        {
            return 0;
        }
        return pixels[static_cast<size_t>(y) * width + x];
    }
    void fillRect(int x, int y, int w, int h, uint32_t rgb)
    {
        int x0 = std::max(0, x);
        int y0 = std::max(0, y);
        int x1 = std::min(width, x + w);
        int y1 = std::min(height, y + h);
        for (int py = y0; py < y1; ++py)
        {
            for (int px = x0; px < x1; ++px)
            {
                setPixel(px, py, rgb);
            }
        }
    }
    void drawGradient()
    {
        for (int y = 0; y < height; ++y)
        {
            for (int x = 0; x < width; ++x)
            {
                uint8_t red = static_cast<uint8_t>(255.0 * x / std::max(1, width - 1));
                uint8_t green = static_cast<uint8_t>(255.0 * y / std::max(1, height - 1));
                uint8_t blue = 100;
                uint32_t color = (static_cast<uint32_t>(red) << 16) |  (static_cast<uint32_t>(green) << 8) | blue;
                setPixel(x, y, color);
            }
        }
    }
    void present()
    {
        /*
         * Convert our RGB framebuffer into the format expected by X11.
         *
         * XPutPixel is intentionally used here because it handles the
         * current X11 visual correctly. It is simple but not especially
         * fast. Later, you could optimize this using XShm or OpenGL.
         */
        for (int y = 0; y < height; ++y)
        {
            for (int x = 0; x < width; ++x)
            {

                uint32_t rgb = getPixel(x, y);
                unsigned long x11Pixel = ((rgb >> 16) & 0xff) << 16 | ((rgb >> 8)  & 0xff) << 8  | (rgb & 0xff);
                XPutPixel(image, x, y, x11Pixel);
            }
        }
        XPutImage(display,window,gc,image, 0, 0, 0, 0, width, height);
        XFlush(display);
    }

private:
    void destroyImage()
    {
        if (image)
        {
            XDestroyImage(image);
            image = nullptr;
        }
    }
};
//endregion

int main()
{
    Display* display = XOpenDisplay(nullptr);
    if (!display) {return 1; }
    int screen = DefaultScreen(display);
    XID Xid = RootWindow(display, screen);
    Window window = XCreateSimpleWindow(display, Xid,100,100,800,600,1, BlackPixel(display, screen), WhitePixel(display, screen));
    XStoreName(display, window, "My C++ Canvas Framework");
    XSelectInput(display, window, ExposureMask | KeyPressMask | StructureNotifyMask );
    Atom deleteWindowMessage = XInternAtom(display, "WM_DELETE_WINDOW", False);
    XSetWMProtocols(display,window,&deleteWindowMessage,1);
    XMapWindow(display, window);
    GC gc = DefaultGC(display, screen);
    Canvas canvas(display, window, gc, 800, 600);
    bool running = true;
    while (running)
    {
        XEvent event;
        XNextEvent(display, &event);
        switch (event.type)
        {
            case Expose:
            {
                canvas.present();
                break;
            }
            case ConfigureNotify:
            {
                int newWidth = event.xconfigure.width;
                int newHeight = event.xconfigure.height;
                if (newWidth != canvas.getWidth() || newHeight != canvas.getHeight())
                {
                    canvas.resize(newWidth, newHeight);
                    // Redraw after resizing.
                    canvas.drawGradient();
                    canvas.fillRect(100,100,250,150,0xff0000);
                    canvas.present();
                }
                break;
            }
            case KeyPress:
            {
                KeySym key = XLookupKeysym(&event.xkey, 0);
                if (key == XK_Escape) {  running = false;  }
                break;
            }
            case ClientMessage:
            {
                if (static_cast<Atom>(event.xclient.data.l[0]) == deleteWindowMessage)
                {
                    running = false;
                }
                break;
            }
        }
    }
    XCloseDisplay(display);
    return 0;
}

RDP Sessions in Linux are not actually that well documented - but if you run a lot of headless servers - you probably will eventually need it, and  it is nice to have a handy reference guide around to setting one up.  We will look at three quick parts to the RDP. We also will keep this guide short - we are all busy!

• Installing the RDP Server using apt install and then activating it  using systemctl
• Installing the RDP Client and getting a connection going
• Tunnelling a RDP through a ssh for very secure over the internet desktops.

Let's get started!

Part A. Installing The RDP Server

Install your supports

sudo apt update
sudo apt install xfce4 xfce4-goodies xrdp xorgxrdp -y

Configure xfce4 for your any user that will use it.

echo "startxfce4" > ~/.xsession
chmod +x ~/.xsession

Once you have this done - you will need it to start automatically, so:

sudo systemctl enable --now xrdp

And then make sure it is working:

systemctl status xrdp

It will look like this if it is good:

Is it running? Well let's check:

nmap 192.168.1.3

Yes, atypically it binds to 3389 and you can see the port is open.

Part B. Installing  / Running  XFreedp3 for a Remote Desktop

• This is your client that will connect to the remote desktop that is waiting. In ParrotOS it is already built in - and with many Linux distros, so open a terminal and just type xfreedp3 and see if something shows up.

sudo apt install flatpak
flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak install flathub com.freerdp.FreeRDP
flatpak run com.freerdp.FreeRDP

Remote Connecting to a client desktop.  

• Atypically it will resort to a very small desktop so these settings are suggested:

xfreerdp3 /v:192.168.1.3 /u:<user> /size:1920x1080

It will ask for Domain <hit enter> and then enter your user password:

Part C. A Secure SSH to RDP (And Closing Port 3389)

• Over the internet is an entire different game, and people are probably going to want a lot more security on their connection.  So first steps  - bind RDP only to local host, inside /etc/xrdp/xrdp.ini change the bindings from your external to internal ip address 127.0.0.1

[Globals]
address=127.0.0.1
port=3389

Next you want to make a Local tunnel to the now internal only RDP port, so in ssh it would look like the following:

ssh -N -L 3390:localhost:3389 <user>@<ip>

However you may need to expand this as:

ssh -X -L 192.168.1.64:3390:localhost:3389 c@192.168.1.3

This command is rare, and often very poorly documented on the internet so we will explain it in detail.

• -X (Allow graphical trunking). It can be debated if this is required for an RDP..
• -L (Locally a port will be opened)
• 192.168.1.64:3390  This is the local port that we are binding to where we are working if we wanted to bind only to a local port we could change it to 127.0.0.1:3390
• localhost:3389 This is not always intuitive but this is where the bind will sit on the remote machine.
• c@192.168.1.3  - Why are we adding this if we are specifying the localhost:3389  - this is the account on the remote machine that will permit the localhost:3389

Please note if you need more training and tutorials on how powerful SSH (how it can pipe mysql / html / anything) check out our tutorial:

ssh workups - plumbing and piping - tunneling, reverse tunneling, socks5, ssh jumps and funsies.

ssh workups - plumbing and piping - tunneling, reverse tunneling, socks5, ssh jumps and funsies.

Hot Configthinkmeltprotonmail.com

Finally now that the tunnel is open we then pipe xfreedp3 through it with:

xfreedp3 /v:192.168.1.64:3390 /u:c /size:640x480

In this instance it will ask if we trust the remote certificate as in:

We now get our little 640x480 desktop

Please note we made no attempt to directly connect to the RDP port, we only connected to our local ssh tunnel.

Speed

• Because you are tunneling through a LOT of encryption it may be very slow, but it will give you secure remote desktops.

FreeToken — Bring Frontier to Edge

Download and run large language models on your own machine — private, offline, free.

FlashML

The LLM World moves fast - so fast that it is a full time job reviewing software... A new  LLM manager was making it's debut, and people were making some very large claims about it. Above is the paper (with it's claims) which looked pretty dramatic - the capability ot run workhorse level LLM's on a 4060 laptop.  Really!?

FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution

Frontier open-weight models are increasingly available, but serving them still largely assumes datacenter infrastructure. We present FreeToken, an edge-native MoE serving system that treats a personal machine not as a small GPU, but as a unified, elastic inference platform. FreeToken co-designs the…

arXiv.orgShuo Yang

FreeToken is fast. Comparing to Ollama, we have 3–4× faster decode, and 6–30× faster prefill

Shuo Yang (@Andy_ShuoYang) on X

FreeToken is fast. Comparing to Ollama, we have 3–4× faster decode, and 6–30× faster prefill How? We introduce bandwidth-adaptive CPU–GPU execution + semantic-aware caching across agent turns. More details in the technical report: https://t.co/V3sIxbPMGU

X (formerly Twitter)

We pulled and installed their Ubuntu image, some issues in that it did not want to work through a ssh console, however for some unknown reason - we noticed the following.

• We built an entire RDP remote desktop, and then used it  to run this software once on the desktop. Once it has run once we noticed that you could again X-11 into it. So note that. Here is our guide we built but found out after we never needed it, as we could of simply just walked up to the headless machine - hooked up a monitor and then got it working, however if you have a truly remote headless machine you sorta have to do this once.

RDP Sessions For All Your LLM Needs

We go over setting up a RDP and then tunneling it through a SSH!

Hot Configthinkmeltprotonmail.com

First time you run it will will present itself as:

It then asks you where you want your models to store:

This is where it got sketchy, we stick to huggingface.com

FreeToken Engine then wants to install itself:

It did efficiently install a number of Nvidia libraries so:

Once it is installed - it interestingly offers a management harness, that both allows you to automatically download and run various models, along with connecting them to online models if you choose.

Apps

• This came with an apps section that looks to install the the large-player connectors and harnesses.

The real 'meat' test was to see what happens when something that out-sized the VRAM significantly was pulled and run - the objective being to see if the claims this 'tuned / management' system was apropos, or hype. So we pulled something that we usually never work with an LLM outside our resource class, along with a couple other models. That was our end-game plan, but we initially did not get there (but we did in the end!)

Double Failure.

• We initially encountered two failures right off the top, and looking at the logs we could see it attempting to compile cuda-utils as in:

Third Try (Using git / uv) SUCCESS

• Next we attempted looking at the git and following it's guidance , to whit it required ft to be installed before you can run freetoken-desktop

GitHub - FlashML-org/FreeToken

Contribute to FlashML-org/FreeToken development by creating an account on GitHub.

GitHubFlashML-org

We installed uv via:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv  # Make a virtual environment for uv
uv pip install "freetoken[accel]"

After this point we had a sort of a mangle setup - you must have ft installed (and referenced) from the command-line, here is how you do it - by activating the ft .venv and then starting freetoken-desktop from a source of it. source ~/.venv/bin/active effectively means 'start and activate this virtual environment that we built with uv pip install "freetoken[accel]" and then we will call freetoken-desktop

source ~/.venv/bin/activate
ft --version  #make sure this is inside your command line
freetoken-desktop   # Through ssh -X or a RDP or direct desktop.

gpt-oss-20b  MXFP4

how are you today? - 0.8 Tokens/s (medium thinking)

What is very different about this system is low, medium high gave all approximately the same speed, as it claims this system is re-balancing on the fly. We were getting about 13.8 - 15 Tokens/s consistently. That was not the speed-ups we were looking for.. or anything close to what they were claiming. Be we did our diligence and checked them out.

A Unique System / Conclusion

• We did not see anywhere to add a custom MCP end-point.
• We did notice something we had never seen any other system do before the ability to re-balance Moe Weights from the CPU to the GPU on the fly - this was very interesting.
• We ran their 'fast' model and as a Moe it should of been blazing but we were looking at about 15 Tokens/s. We tried moving the sliders around and reducing the context - it didn't matter.
• It was challenging to even get working as ft had to be installed from a .venv and from inside that venv THEN you could launch freetoken-desktop but this was not transparent with the creators presuming you would know how to make all this work.
• Conclusion is it's an interesting system with some unique features, and serious work by their team was put into it, it could be suspected it will be a lot better in the future!

Firstly expectations. What do we define as a 'workhorse' and an 'assistant,' well:

• Assistant LLM - Models that run inside a 8-12GB Graphics card and can do powerful lookups and contexts typical under or up to 64K.  These run great inside a 3060ti or a 3080. They use MoE (Mixture of Experts) a non-dense type model.
• Workhorse LLM - Models that run inside a 12-24 GB Graphics card and handle the large contexts (typically up to 256K), and or are capable of working all night without overflow by breaking up the job with Hermes or LLMQP.  They run great inside a 3080 / 4080 / 3090 and higher graphics card.
• Production LLM - Models that run on multiple 3090ti's, DGX Sparx, and Commercial equipment.  That term is debatable as many would only consider push-button get-application like Claude or Fable to qualify.

Token Speed Matters

Be honest with yourself. 4 months ago 10-12 Tokens/s was considered amazing. In reality we can consider the following inference numbers, and just review what pre-fill and generation references even are:

Pre-fill - This is where the LLM reads the passed information, the pre-prompts, and any pasted files etc.  Anything over 75 Tokens/s are acceptable, and even localLLM's have been seen working up to 1000 Token/s depending on the model.

Generation - This is where the LLM will generate tokens or answer questions. A token is basically considered roughly 1-five character word and still loosely defined.

• 5 - 15 Tokens/s. This is actually slow. It's okay for automated processes that run all night on your behalf, but in reality most people if they can type up code faster than the LLM they will find themselves not waiting for it.  Usually the first time a person sets these up they are very excited that they can even get that.
• 15 - 40 Tokens/s. Somewhat better but again it's tolerable but tool calling can feel slow. The real secret to getting speed is running a sparse model that has the 'MoE' or Mixture-of-Experts and has less active parameters than a 'dense.'
• 40 - 100 Tokens/s. This is the sweet spot where the LLM can work as fast as your work flow, and as you need questions.  This is what you want to shoot for in a localLLM at this time where it can effectively work at the speed you want to review it.
• 100+ Tokens/s is really  super flow, it generates on a consistent basis information faster than a human can even manually review.   In this case one would want a harness or a LLM controller. A great case is Hermes, or LLMQP. For instance if you have a 4080 and you run a MTP head and you run a 8B you will probably see a 100 Tokens/s speed.

Advantages

• There are massive advantages to going to a LocalLLM. Firstly your proprietary data will not end up farmed into someone else's next token model.
• Sensitive data is retained such as doctors, lawyers and client-privelege scenarios. Legal requirements can force many to taking this path.
• Inventions and NDA type materials do not need to risk being harvested by the cloud LLM.
• This interview shows how concerned corporate is about going with a private LLM.

Let's start at the most basic actually useful LLM we can build < $500.

Starting at the Very Bottom - A Epyc 3151 Build (15,000 Specmark for $61). Then Add a Used GPU off FBM.

• 4 cores / 8 threads / 2.9 Ghz and a PCIe 3.0 16 GB/s for the cost of a Raspberry Pi! What's not to love!
• Add up to 256 GB DDR4 RAM.

• Next we want to add a low-cost GPU.  The best bang-for-buck right now is the 12 GB RTX 3060 if you can find one..

The key here is to find the first models that have Tensorcores, and also has 12 GB VRAM. This is key! You can absolutely get by on the 8GB VRAM - for an assistant, but if you want to get to a 'workhorse' level LLM - you want to try to find a 12 GB or higher VRAM. 16 GB VRAM like the 4080 has will be preferable, however it looks like their current price at the time of this writing has pushed up their prices significantly into the $1200 range. If you have hundreds to spend then of course buy a 24 GB 3090 but expect to pay lots right now as they are selling upwards of $1500 as of mid-2026.

You can run very capable 8GB models and we proved it.  Naturally however we find that once you start sending large code bases the desire for the larger context > 128K becomes immediate.

Game Changer! Crash-Out! Good Production on a Ryzen 5 2600 (6-core/12thread AMD) w 3060ti/8GB VRAM

Crash-Out! Good Production on a Ryzen 5 2600 w 3060ti/8GB VRAM. We showed you can actually get very powerful productive capability on a 3060ti!

Hot Configthinkmeltprotonmail.com

If you get to the 12 GB model or higher - it then allows you to run large contexts. If you can find a 16GB (I know they are getting more expensive by the day) you can run the 128-256K Contexts. Turboquant is really key here.

Here is a table of possible configurations for lower-end compute. The key here is that they all have the Tensor Cores, and that is going to make a dramatic performance boost. Of interest is the large jump in Tensorcores on the 3080 plus they have the 12GB, but they were never really noticed as a LLM card - so they have stayed relatively cheap.  It's a perfect 'sweet spot' to go with the Epyc 3151 node if you are trying to build for under $500.

Keys are in the Tensor Cores, Problem is in the PciE Speed.

• 8 -9 B sized models will run inside a 8 GB model. We showed they are very capable with these kinds of setups. Sparse models lowered the active parameter counts and they can run really well on mixed layer loading. This is where some models load to the CPU and some load to the GPU.

Game Changer! Crash-Out! Good Production on a Ryzen 5 2600 (6-core/12thread AMD) w 3060ti/8GB VRAM

Crash-Out! Good Production on a Ryzen 5 2600 w 3060ti/8GB VRAM. We showed you can actually get very powerful productive capability on a 3060ti!

Hot Configthinkmeltprotonmail.com

We also showed if you can find a 16 GB VRAM or higher and you have a TensorCore enabled card - you can get an incredibly good 'Workhorse' level LLM that is even fast and has a large context. For instance:

Qwopus-3.6-35B-A3B-Coder Review. A Powerful LocalLLM Tuned for Coders. It is one of the best 35B sized models we have ever reviewed. Seriously.

Qwopus-3.6-35B-A3B-Coder is a localLLM dream for software developers. Incredibly accurate and powerful prompt processing!

Hot Configthinkmeltprotonmail.com

Why Was the 3090 Dual with NVLink so Popular over say Dual 4080's?

• Anywhere the model was split (and was a 'dense' model) it required massive bandwidth between the layers. If the model had to communicate over a PCIe 3.0 bus even at 8 GB/s bandwidth (16 GB/s) it would kill the model speed.  The NVLink was incredibly popular because it enabled up to 112 GB/s with an interlink card directly connecting two GPUs independent of the PCIe.  The only other option is to try to find a model that fit completely inside the GPU.
• The other option naturally was anything with a unified memory which implied that the RAM of the device shared with the GPU, and worked at dramatically higher speeds not needing the very slow PCIe bus.

People Started Trying to Run LLM's on Everything

Even a old mining rig the BC-250 because of it's very fast unified memory speeds of 448 GB/s plus the fact that it sells for less than a single 3060ti made them very good candidates for the 8B platform.

Loktar 🇺🇸 (@loktar00) on X

This is actually CRAZY!!! Using llama.cpp RPC I have 2 BC-250’s setup so far, they’re able to run Qwen 27b at Q4, and 35b at Q4 as well. This is without extra CUs unlocked: Qwen 27b with MTP - 14.5 tk/sQwen 35b with MTP - 47 tk/s For $300 I’m getting these speeds! This is

X (formerly Twitter)Loktar 🇺🇸

Exploring the Older Nvidia Tesla / Quadro Etc.

• People even recompiled various LLM's so that they could run somewhat quickly on very older hardware, like the K80, and reported 11 Token/s (showing again even industrial older cards did not hold up as well as any cards that had Tensorcore.)

Chicken Little (@chkn_little) on X

@LottoLabs he wrote me today that he’s running Qwen3.6-35B-A3B-UD-IQ4_XS on the K80 using llama.cpp at 10 tps can a add the k80? I think it’s worth an entry

X (formerly Twitter)Chicken Little

The DGX Spark (Nvidia's LocalLLM Solution)

• At the time of this writing they haul heavy in the price tag - and the market demand confirms why. At $4679 on amazon.com their compute capability comes in at a whopping 1 PFLOP (1000 Trillion Floating Point Operations/s) + 20 core main compute + 128GB LPDDR5X. Because of their unified bandwidth they can send back and forth to the memory and the GPU without requiring going out to a PCIe lane, and this is fast coming in at 273 GB/s.  Please note B and b matters - if you are talking 10 Gbit - that sounds fast, but divide it by 8, then it comes in at 1.25 GBytes/s.  This is why nobody is running inference over the internet - well at least not yet..
• They could be interconnected at very high speeds allowing very large distributed models to split between them. The speed of the interconnect of the ConnectX-7 coming in at 200 Gb/s (Giga-bit/s)

The MAC Mini Pro Weighed in

• On the Macintosh side - due again the very fast interconnection capabilities between machines allowed for stacked units to run very large models quickly and capably.
• Demand was so high there were literally none for sale on Amazon with only second-hand sellers moving them on ebay.
• We do not really cover them simply in that they are not even available. 3-4 of them will cost as much as commerical H100 units etc.

Bus BandWidth Was Always What  Really Mattered

We can see that until PCIe 6.0 we won't even begin to match the speeds of the NVLink, consider the following chart.

The bandwidth always significantly mattered. Tools existed exactly for diagnostic testing, and if you do want to get into the meaty details of measuring GPU-to-GPU bandwidth, PCI-e-to-GPU, GPU-to-Itself you can compile and install the mbw tool:  This is where you find out what stuff really can do - not the brochure sale..

Linux/LLM Speed Bandwidth Testing mbw / nvbandwidth

The #1 limiting factor if your LLM will run quickly or not is memory bandwidth. Memory Bandwidth of your RAM.Memory Bandwidth of your GPU.How quickly data can get between the GPU and the CPU over your very slow PCIe bus.Memory Bandwidth of your RAM. In Linux we

Hot Configthinkmeltprotonmail.com

This is where many people then sought out 'server' level motherboards in that they had the proper PCIe 4.0 x 7 such as the TRX40.  By getting up to 4.0 they could theoretically get better performance or run 4-8 card configurations. Used Epyc Motherboards were a good fit.

Re-tuned Moe + MTP + TurboQuant + MCP Was the Game Changing Solution

People started looking at all kinds of ways to speed things up, make them smaller, make them faster, and or make them more accurate.  What were all these different additives? Well when they all were added up they made a dramatic and powerful set of tools that reduced cache size, increase speed, and enabled tooling.

MoE - Mixture of Experts.  This allowed sparse models to have much less active parameters.  By doing this it greatly reduced the CPU/ GPU load, and allowed for the offloading and powerful fitting llama.cpp configurations such as:

--override-tensor "\.ffn_.*_exps\.weight=CPU" \

TurboQuant - kv_cache expanded quadratically, long contexts would destroy the VRAM and kept the hardware requirements very expensive. A full github and guide to install it. It was added very quickly to llama.cpp, and this brought the 'Workhorse' level LLM into existence! Prior to TurboQuant the small consumer was never really a play in this.

MCP - Model Context Protocol allowed for powerful tool calling. The right tool calling allowed for LLM's to correct their work. Over and over until they get it right. Here are ten+ powerful MCP's that are all opensource!

MTP - Multiple Token Prediction came next. By running multiple parallel heads of certain layers it could see significant speedups in some instances up to 200%!

Re-Tuned Models - By taking strong base models and carefully applying extra token layers people were able to get significant boosts in accuracy and performance of an LLM in a benchmark.

The Results - You are the winner

It created models that were  so powerful, capable and run on minimal hardware that the industry has not even fully realized how capable a localLLM is. Consider a 35B that can one-shot an entire Asteroids game, accurately, and cleanly and run on a 4080, it is right here. Again it only works because all of the above features are working at the same time.

Qwopus-3.6-35B-A3B-Coder Review. A Powerful LocalLLM Tuned for Coders. It is one of the best 35B sized models we have ever reviewed. Seriously.

Qwopus-3.6-35B-A3B-Coder is a localLLM dream for software developers. Incredibly accurate and powerful prompt processing!

Hot Configthinkmeltprotonmail.com

Or we show again how you can run capable models inside a 3060ti inside a 8GB VRAM.

Game Changer! Crash-Out! Good Production on a Ryzen 5 2600 (6-core/12thread AMD) w 3060ti/8GB VRAM

Crash-Out! Good Production on a Ryzen 5 2600 w 3060ti/8GB VRAM. We showed you can actually get very powerful productive capability on a 3060ti!

Hot Configthinkmeltprotonmail.com

Conclusion - Everyone Wants a LocalLLM. Configurations Matter.

You are not going to replace a $20 Billion dollar server farm with racks of H200 inference engines with a $1200 4080 Nivdia RTX you scored off FaceBook.  That's just not realistic. But configurations really matter. The right model, the right tuning, the right MCP agents they are capable, very quick, reliable and you never have to worry about your research being harvested by a corporate LLM.   You can take odd paths with the B-250 but find you outgrow a limited memory device, and find yourself going big for a DGX Spark. Corporate interests using tokenMaxxing budgets and getting billion dollar builds realized themselves even -  they jumped their own shark, and were getting massive Token bills. Suddenly Elon was limiting employees to $200/week, Meta was scaling things back, and it was becoming quickly public  everybody didn't want the prompt harvesting companies taking their proprietary data and using it to train the next 6T model.

• Commit the time. It can take a few days, but just learn a little bit at a time. If you would like a world-class LLM that encorporates all the bells and whistles start with this StudentLLM.
• This site is utterly CHOCK full of opensource guides for MCP tools, for various llama.cpp configurations, model reviews etc. It is a win for you!