It is quite nice, but not as nice as you'd want. You still have to set platform specifics when running completions when you want to tune things like temperature, effort, max tokens, etc.
I'm one guy doing it for free. Happy to see your contribution!
techscruggs•Jun 24, 2026
And thank you! It is absolutely awesome and a true joy to work with.
mosselman•Jun 24, 2026
Hi! Valid challenge, I am probably misremembering. We were playing with various 'one-interface to all providers' solutions and I might have mixed up RubyLLM there. Sorry for that.
I will have a deep dive into which things I felt we needed to adapt per provider.
I didn't mean to imply that you have to solve all of our wants of course.
One thing we did do was monkey-patch the spot where tool_calls are performed by RubyLLM. We had our own mechanism for that and were able to skip RubyLLM's and still extract the tool calls and run them through our own tool harness. That all worked beautifully. I don't know if that type of stuff is something you want PRs on or that you want to keep steering towards the route that does everything within RubyLLM classes. Happy to contribute some of that.
earcar•Jun 24, 2026
Interesting! What were you guys trying to achieve by running them in your own tool harness?
swe_dima•Jun 24, 2026
I found Ruby LLM to be surprisingly good - in terms of usability it's close to Vercel's AI framework.
It tries to strike a balance between working out of the box and being flexible... which has its challenges, still nice overall.
One big real-life pain I experienced is that caches don't always work, e.g. for xAI, since it only supports completions API and thought signatures are returned wrong.
earcar•Jun 24, 2026
Thank you!
Responses API is now implemented and it's coming in RubyLLM 2.0
RubyLLM is very easy to use. Made extensive use of it for a project last year. Drawbacks are it was difficult to instrument for true trace observability and it has a pattern where retries will delete the underlying models so the history you see is clean but not necessarily great for seeing exactly what the sequence of API calls was.
I have an open source gem called Raix that builds on top of RubyLLM's abstractions and is quite popular. https://github.com/OlympiaAI/raix
notpachet•Jun 24, 2026
Why would anyone still build in dynamically typed languages in 2026? Why relinquish the crystal clear signals that static typing is able to provide to the LLM?
jimbokun•Jun 24, 2026
This is not a tool for using LLMs to write Ruby code.
taylorlapeyre•Jun 24, 2026
Well, LLMs have an obscene amount of context built into their weights about Ruby on Rails, and can work within it extremely quickly.
lackoftactics•Jun 24, 2026
Even as rails dev, I am seeing that you might be right. It’s really hard to find specific pros nowadays that Ruby brings to the table. All that talk about conventions over configurations and vast presence of Rails in weights is fun, but if writing speed isn’t an issue anymore, then Ruby on Rails has serious problems with larger codebases
digitaltrees•Jun 24, 2026
We use this in production for a few apps. Great project.
meerita•Jun 24, 2026
"What is the best language in the world (say ruby)" ;)
rohitpaulk•Jun 24, 2026
We use RubyLLM in production too, the most elegant library in this space I've seen so far.
I also liked how they run the issue tracker. If you select "Feature Request", it makes you explain how you explored workarounds, why you believe it belongs in RubyLLM etc to prevent scope creep.
hit8run•Jun 24, 2026
Using RubyLLM in production for https://usetix.io
It drives our event chat agent that is enhanced with toolcalls etc. Super happy with it.
aniokono•Jun 24, 2026
I haven't tried it but it looks promising.
arbirk•Jun 24, 2026
I have been a fan of Ruby for many years, but in this fast paced era the Ruby ecosystem always struggled with the dependency versioning. Gems I relied on were never available or compatible with the rest of the ecosystem.
MitziMoto•Jun 24, 2026
We use and love RubyLLM! A wonderful and easy to use framework.
Agreed with another commenter on the frustration with the responses API not being naively supported; that seems like a huge miss. There is a connector from another dev, but it's buggy and not as high quality as the main gem.
Really looking forward to future development and especially 2.0!
Edit: Just saw that responses API is now native? I will definitely check that out.
17 Comments
I'm not sure where you got that.
`chat.with_temperature(0.2)`
https://rubyllm.com/chat/#controlling-response-behavior
`chat.with_thinking(effort: :high, budget: 8000)`
https://rubyllm.com/thinking/#controlling-extended-thinking
Max tokens is the only one of your list that require provider specific params:
https://rubyllm.com/chat/#provider-specific-parameters
I'm one guy doing it for free. Happy to see your contribution!
I will have a deep dive into which things I felt we needed to adapt per provider.
I didn't mean to imply that you have to solve all of our wants of course.
One thing we did do was monkey-patch the spot where tool_calls are performed by RubyLLM. We had our own mechanism for that and were able to skip RubyLLM's and still extract the tool calls and run them through our own tool harness. That all worked beautifully. I don't know if that type of stuff is something you want PRs on or that you want to keep steering towards the route that does everything within RubyLLM classes. Happy to contribute some of that.
It tries to strike a balance between working out of the box and being flexible... which has its challenges, still nice overall.
One big real-life pain I experienced is that caches don't always work, e.g. for xAI, since it only supports completions API and thought signatures are returned wrong.
Responses API is now implemented and it's coming in RubyLLM 2.0
https://github.com/crmne/ruby_llm/blob/main/lib/ruby_llm/pro...
I love how MINASWAN Hacker News is when talking about Ruby!
Rails-style instrumentation landed in 1.16.0.
https://rubyllm.com/instrumentation/
https://github.com/tweibley/legate
I also liked how they run the issue tracker. If you select "Feature Request", it makes you explain how you explored workarounds, why you believe it belongs in RubyLLM etc to prevent scope creep.
Agreed with another commenter on the frustration with the responses API not being naively supported; that seems like a huge miss. There is a connector from another dev, but it's buggy and not as high quality as the main gem.
Really looking forward to future development and especially 2.0!
Edit: Just saw that responses API is now native? I will definitely check that out.