Introduction
During my usual YouTube browsing, I stumbled upon a video (here) showcasing an AI tool called Fabric, a Python utility that facilitates interaction with AI models. The demo focused on OpenAI's paid version, but I was eager to explore its potential using a local LLM model.
Understanding Fabric
Fabric is a Python tool that takes text input and submits it to AI models via an API endpoint, along with an advanced prompt.
It includes a library of pre-built advanced prompt templates called patterns. These patterns can be customized or created from scratch to fit individual needs.
These two features make Fabric a unique tool. It acts as middleware between the user and the AI engine, allowing anyone to tailor their interactions with AI models, making it easy and efficient to use.
Fabric
To install Fabric, follow the guide on the GitHub page: https://github.com/danielmiessler/fabric.
Once installed, you can run basic commands like fabric --listmodels to list all available models.
If you want to connect to OpenAI, you can run fabric --setup. It will run the initialization phase and ask for your API keys.
However, if you don't want to pay for an OpenAI key, you will need to run your model locally. This is where LM Studio comes into play.
LM Studio
LM Studio is a free tool that allows users to download and run models locally on their machines. While it may be slower than using the paid version of OpenAI, it provides an opportunity to test LLM without incurring costs. To use LM Studio, visit https://lmstudio.ai/docs/welcome and install it.
Integration
Now that you have both Fabric and LM Studio installed, let's connect them.
To integrate Fabric with LM Studio, follow these steps:
Download a model from LM Studio (I tried llama2, llama3, and Gemma, but stopped at llama3 as it's the latest and most advanced model available).
Run the LM Studio local server. Increase the context length to the maximum and use your GPU if you have one.
You should end up with something like this:
Configure Fabric by setting the following environment variables:
export OPENAI_BASE_URL=http://localhost:1234/v1
export DEFAULT_MODEL=Meta-Llama-3-8B-Instruct-Q8_0.gguf
export OPENAI_API_KEY=lm-studio
That's all. You can now test if Fabric is properly configured with LM Studio by running:
fabric --listmodels
It should print the model you've loaded in the LM Studio server.
You can also try to send your first and most important request to Fabric using a generic AI pattern with the following command:
echo "what is the meaning of life" | fabric -sp ai
Using Fabric
Pre-built patterns
Patterns are advanced, customizable, and shareable AI prompts. Fabric comes with a library of pre-built patterns that you can use and modify to suit your needs.
You can list all the available patterns with the command fabric -l.
Here are some patterns I particularly like:
extract_wisdom: Extract important information from any text source (blog post, YouTube video transcript, PDF, etc.).
ai: A generic pattern for standard requests.
write_pull-request: To write beautiful PR/MR without effort.
agility_story: For creating agile user stories.
provide_guidance : Your AI psychologist
Writing Your Own Patterns
To demonstrate this capability, I created a custom pattern to improve this blog post. Yes, what you're reading has been generated using the process I'm about to explain – isn't it amazing?
If you look at the pattern library, you'll find a particularly useful pattern called improve_prompt. This pattern will help you write your own patterns.
Let's try creating a technical blog post pattern:
echo "As a technical writer, you take draft content as input and engaging technical guides in the form of blog articles. You provide technical context, explanation, and reference. You write articles of around 2000 words. You provide clear and detailed information. You use simple examples to explain concepts. You provide external links and sources. You provide actionable articles with a mix of theory and tutorial to help people understand concepts by putting things into practice. You write in a balance professional and casual way." | fabric -sp improve_prompt
Here is a snapshot of the produce pattern
Take the output and save it as a file under /Users/<user>/.config/fabric/patterns/write_tech_blog/system.md.
This prompt is ready to be used, improved, or customized.
Now, if you list your models with Fabric -l, you should see your new pattern.
To pass some data to Fabric, you can use several methods:
To test our new pattern, gather some draft ideas for a blog post and pass them to Fabric with a command like pbpaste | fabric -s -p write_tech_blog.
This should give you pretty good articles that you can use right away or customize and improve yourself (or with other patterns).
If you want something different or more detailed, review the pattern file or regenerate it by changing the input of the initial improve_prompt process.
Crafting prompts is almost an art and definitely an iterative process. To improve your prompt crafting skills, I strongly recommend taking this free course: https://learn.deeplearning.ai/courses/chatgpt-prompt-eng
Use multiple patterns
Another great feature of Fabric is its ability to use the output of one Fabric process as input for another. For example, you could extract wisdom from an article and use that to create a Keynote presentation, like this:
pbpaste | fabric -p extract_wisdom | fabric -s -p create_keynote
This will obviously be slower because it runs Fabric twice, but I found this concept of data refinement very powerful and full of potential.
Here is a snapshot of the result for this blog post :
Conclusion
Fabric and LM Studio offer a powerful combination to seamlessly use the AI platform while extracting the most value from it. By creating custom prompts and patterns, users can fully tailor their interactions with the AI model to suit their needs.
I strongly believe that AI tools are here to extend our capabilities and help us achieve more, faster. These tools have the potential to be as transformative as the industrial revolution, which fundamentally changed the way we work by introducing machines that could perform tasks more efficiently than humans. Similarly, AI will not only replace some jobs but also create entirely new ones that require different skills and competencies.
For now, these tools are still in the early stages and sometimes produce inappropriate or vague content. This is why the output of any AI-powered writing tool should always be reviewed and adapted by humans (for now :)).
Key Takeaways
Fabric is a Python tool that interacts with API AI models and injects custom prompts called patterns.
Patterns are pre-built templates that can be customised to meet individual needs.
Writing your own pattern can help you tailor your interactions with the AI model.
LM Studio is a free tool that allows users to download and run models locally on their machines.
