Web-Performance Blog
This is also a show-and-tell of options on how to use and play with Ollama. I tend to use my Ollama as a headless system sending requests via VPN for remote processing on my Ollama macOS.
I’ll help you create a persistent configuration for OLLAMA_HOST on macOS. The most reliable way to do this is to create a LaunchAgent plist file.
I’m collecting a multitude of skills in local AI processing, and today it is audio to text. I switched to Ollama for most of my AI processing needs, but audio files can’t yet be processed via Ollama APIs. Therefore, I was looking for an alternative, however, I didn’t look for a RETS API; running a Docker container will be sufficient for me.
I used whisper on Docker.
As I work every day, I constantly see recurring work that I would like to optimize and automate. There are use cases that feel obvious that they should work and easy to do, but the real-world experience is different. Read about my learning.
Ollama is great for running a local self-hosted AI REST API. You can load all kinds of models, whether they’re for chatting, vision, or embedding. At this time, I’m not aware that voice models can be used for input or output, but that might only be a matter of time.
Why would you like to run Ollama? First of all, the models you can load are likely less powerful than what you can consume from the OpenAI API or Anthropic API. However, you can fully locally process, gaining the privacy of not having to process in a US datacenter but rather it does not even have to leave your house. Additionally, if you need to process massive amounts of data and the precision is sufficient from the open models, you might have to wait longer for all to be processed, but using public APIs can also rack up vast amounts of cost to do so. Even by error, you might burn through $80 in no time when you missed a error catch in your code. In the Ollama case, you only have your upfront cost for the device and the energy burned. But then you have no sudden surprises. But also your processing is of course limited; you can’t massively parallel process. So it highly depends, but there is a sweet spot to solve issues on your own machine.
DeepSeek’s first generation reasoning models with comparable performance to OpenAI-o1.
In the words of OpenAI.
A new series of AI models designed to spend more time thinking before they respond. As AI becomes more advanced, it will solve increasingly complex and critical problems. It also takes significantly more compute to power these capabilities.
https://openai.com/o1/ (2025-01-22)
Or as an excerpt from Wikipedia.
It spends time “thinking” before it answers, making it better at complex reasoning tasks, science and programming…
https://en.wikipedia.org/wiki/OpenAI_o1 (2025-01-22)
Ahref has released on their website a tool to have an image processed and returning alt text. Processing images is a good way of utilizing AI in my opinion. The offer is nice as an idea but also convenient as the tokens for image processing are rather expensive. Based on the short texts returned, I imagine they resize the image to smaller before having it processed. Try it out and consider using it for your online posting and have the web more accessible.
A recent study by The Guardian found that AI search engines like ChatGPT Search can be manipulated by hidden information on websites.
A variety of products have been introduced to detect whether text is generated by a LLM. However, this is a challenging problem, and in my experience, the attempts I’ve explored have performed poorly. I was able to generate text that was not flagged, while other text I typed received false positives. Consequently, I believe these tools are not yet reliable.
LM Studio is a user-friendly desktop application designed for exploring local and open-source Large Language Models (LLMs). Your data remains securely stored on your machine, and the best part is, it’s completely free for personal use.
I started playing with Claude.ai and this is what I learned.
As is typical with my blog posts, this is a developing story, and I will continue to update this post as I play and learn more. For now, I am still impressed by the quality of the answers. Particularly, coding appears to yield far superior results compared to ChatGPT 3.5, ChatGPT 4, or ChatGPT 4.