jayhemnani
say hi →

/oss · 17 merged

AlexsJones/llmfit

A command-line tool that tells you which AI models will run on your computer's hardware.

the project on GitHub (opens in a new tab)

  1. 20 Oct 2026

    Llama 3 models no longer show double their real context size

    What was broken
    Llama 3.2 models showed a context window of 262144 tokens, twice the real 131072.
    What I changed
    For this kind of model the tool now keeps the context size the model states instead of multiplying it.
    Why it matters
    Fit estimates for Llama 3.2 models start from the real number.

    see the code, #1167 (opens in a new tab)

  2. 20 Oct 2026

    Unknown model size labels are now refused with a clear error

    What was broken
    The claim command accepted any size label. A mistyped or lowercase one was priced like a small default, so the claim asked for far less hardware than the model needs.
    What I changed
    The command now checks the label and stops with an error that names the labels it knows.
    Why it matters
    A claim can no longer be scheduled onto a machine that cannot run the model.

    see the code, #1162 (opens in a new tab)

  3. 19 Oct 2026

    A mistyped memory size is an error, not a silent fallback

    What was broken
    If you mistyped the memory or RAM size you wanted to test with, llmfit only printed a warning and then scored your own machine, while still reporting success.
    What I changed
    I made it refuse a size it cannot read, the same way it already refused a bad percentage.
    Why it matters
    Scripts and tools that read llmfit's results never get numbers for the wrong machine.

    see the code, #1160 (opens in a new tab)

  4. 18 Oct 2026

    The server no longer deletes files at its socket path

    What was broken
    llmfit's server can listen on a local socket file. Before starting, it deleted whatever was at that path, so a typo could wipe a real file and a second server could cut off a running one.
    What I changed
    It now removes the path only when it is a leftover socket nothing is listening on. A regular file or a live server makes it stop with a clear message instead.
    Why it matters
    A wrong path or a second launch can no longer destroy a file or cut off a running server.

    see the code, #1152 (opens in a new tab)

  5. 17 Oct 2026

    Speed targets for mixture-of-experts models use the right size

    What was broken
    llmfit can say how much memory speed a device needs to run a model fast enough. For mixture-of-experts models it counted the whole model, so it asked for far more speed than any device has.
    What I changed
    I made it count only the part of the model each word reads, the same way its speed estimate already does.
    Why it matters
    The speed target for these models is now one a real device can meet, and it agrees with llmfit's own estimate.

    see the code, #1150 (opens in a new tab)

  6. 15 Oct 2026

    Planning and downloads work when the server runs on a local socket

    What was broken
    llmfit can run as a small server that only the same machine can reach. In that mode, its planning and download features, and the dashboard panels that use them, failed with a server error every time.
    What I changed
    I made those features accept a local socket connection as coming from the same machine.
    Why it matters
    The dashboard's planning and download panels now work in that mode too.

    see the code, #1146 (opens in a new tab)

  7. 15 Oct 2026

    Memory needs for DeepSeek-style models are no longer hugely overstated

    What was broken
    Some newer models, like DeepSeek-V3 and Kimi-K2, keep their working memory in a compressed form. llmfit sized it as if uncompressed, about 7 to 25 times too big, so it said they would not fit.
    What I changed
    I made it size that memory the compressed way these models store it.
    Why it matters
    These models are now rated on what they really need, so people with smaller cards can see they fit.

    see the code, #1145 (opens in a new tab)

  8. 15 Oct 2026

    Mixture-of-experts models get memory advice for the size it suggests

    What was broken
    llmfit tells you which AI models fit your graphics card. For one kind of model, it measured memory at one compression level but recommended another, so a model it called a good fit could be too big.
    What I changed
    I made it measure memory at the same compression level it recommends.
    Why it matters
    The memory number and the verdict now describe the download it tells you to get.

    see the code, #1141 (opens in a new tab)

  9. 13 Oct 2026

    Updated model lists now spot mixture-of-experts models

    What was broken
    Some AI models use only a small part of themselves per word. When refreshing its list, the tool spotted these only by an old naming style, so newer ones counted as full-size and got far too low a speed.
    What I changed
    I made the update read the expert counts each model already lists in its own settings file, as the tool's built-in list already did.
    Why it matters
    On an 8 GB laptop, one such model now shows about 147 tokens a second instead of 25, and bigger ones get the memory-saving option they qualify for.

    see the code, #1142 (opens in a new tab)

  10. 12 Oct 2026

    Big models in a fixed format get a correct memory estimate

    What was broken
    Some large models only come in one compressed format. The tool sized them as if a smaller format existed, so it called a model a good fit when it needed far more memory than the computer had.
    What I changed
    I made the tool size these models in the one format they actually ship in, as it already did elsewhere.
    Why it matters
    People no longer download a huge model the tool said would run, only to find it doesn't fit.

    see the code, #1131 (opens in a new tab)

  11. 12 Oct 2026

    Mac-only models no longer show up on other computers

    What was broken
    Some models only run on Apple's Mac chips. The tool spotted them by a word in the name, but one big publisher of them doesn't use that word, so hundreds appeared on Windows and Linux.
    What I changed
    I made the tool also treat everything from that publisher as Mac-only, as other parts of it already did.
    Why it matters
    On Windows and Linux, the list only suggests models that can actually run there.

    see the code, #1130 (opens in a new tab)

  12. 10 Oct 2026

    Upgrade advice now matches the tool's own ratings

    What was broken
    The planning feature graded models with an older, looser rule. It called a model "Perfect" where the main check said borderline, and its "add this much memory" advice fell short.
    What I changed
    I made the planner use the same grading rules as the main check, and sized each upgrade suggestion to reach the rating it promises.
    Why it matters
    Someone deciding whether to buy a graphics card with more memory gets advice that holds up.

    see the code, #1125 (opens in a new tab)

  13. 9 Oct 2026

    The suggested conversation length now fits in memory

    What was broken
    The tool reports how much text a model can hold in mind at once. It sized that to fill all of memory, which its own check calls too tight.
    What I changed
    I capped that figure at the same 98% limit the rest of the tool uses.
    Why it matters
    Following the tool's advice no longer pushes a model past the point where it fits.

    see the code, #1126 (opens in a new tab)

  14. 9 Oct 2026

    The download tip now names the version that fits

    What was broken
    Some huge models keep idle parts in regular memory. For these, the tool picked one compressed version but told you to download a bigger one that didn't fit.
    What I changed
    The tool now reports the version it actually picked, and the memory that version really needs.
    Why it matters
    People download a model their machine can run, not one that is too big for it.

    see the code, #1121 (opens in a new tab)

  15. 8 Oct 2026

    It picks a model size that actually fits your card

    What was broken
    The tool picked the largest compressed version of a model that filled 98-100% of memory, then rated that same pick "Too Tight", even when a smaller version fit fine.
    What I changed
    It now picks only from versions under the 98% limit, so its pick and its rating agree.
    Why it matters
    On a 24 GB graphics card, 124 models went from "Too Tight" to a version that fits.

    see the code, #1120 (opens in a new tab)

  16. 8 Oct 2026

    Search results can now be read by other programs

    What was broken
    Asking the search command for machine-readable output was ignored. It printed a table for humans instead, so scripts reading its output broke.
    What I changed
    With that option, search now returns a proper list of matching models, or an empty list when nothing matches.
    Why it matters
    Scripts and AI agents can use search the same way they already use the tool's other commands.

    see the code, #1113 (opens in a new tab)

  17. 7 Oct 2026

    Typos in filters are now caught instead of ignored

    What was broken
    Mistyping a filter, like "embeddings" instead of "embedding", silently switched the filter off. You got unrelated models back with no warning.
    What I changed
    The tool now stops and lists the valid choices when it doesn't recognize a filter value.
    Why it matters
    A one-letter typo no longer gives you misleading recommendations.

    see the code, #1114 (opens in a new tab)

← every project