AI Experiences

The thread presents a first-hand experience from a founder of an LLM fine tuning startup, highlighting that fine tuning LLMs is as complex as deep learning and poses barriers to broader adoption. Key insights include that fine tuning demands high expertise, often making it practical only for specialized teams or large customers due to economic constraints. Participants discuss that general purpose large models with advanced prompting and tooling often outperform fine tuning in both ease and effectiveness. There is consensus that fine tuning might save inference costs in niche scenarios but struggles with silent failures and complex debugging. The importance of picking the right market segment, such as sovereign compute or deep integration for a single major client, is emphasized, as well as the risk of market fragmentation and difficulty scaling fine tuning beyond expert ML teams.

The original poster shares their experiments with using LLMs to create dynamic, creative text adventure settings, highlighting the promise of LLMs to enhance narrative depth and player freedom beyond traditional scripted games. Replies discuss different use cases like running player characters rather than the game world, challenges of consistency across playthroughs, and the immersion-breaking aspects of LLM-driven storytelling. Insights include considering LLM-driven characters for playtesting TTRPG modules and awareness that the game experience may vary between players due to the generative nature of LLMs. Overall, LLMs show promise as a creativity amplifier, but developers should address narrative consistency and player immersion to optimize the experience.

The original poster shares a practical experience of using an AI code generator (referred to as 'vibe coding') with JavaScript and TypeScript typings, highlighting blindspots in the AI's understanding of outdated type definition packages causing runtime and type checking errors. The discussion reveals that the AI blindly trusts the typings library without cross-verifying its currency or correctness, leading to fragile code and runtime failures. A respondent adds that for simpler packages, LLMs can sometimes successfully infer typings from untyped libraries, suggesting varying levels of LLM effectiveness depending on package complexity. The key insight is to maintain human oversight when working with AI-generated code and third-party typings, especially for older or less maintained packages.

The original poster shares a method of enhancing their blog writing by outlining first and using a large language model (LLM) as a developmental editor to refine the throughline before drafting the full piece. This approach helps separate structural development from voice and sentence-level writing, which they find beneficial. A respondent comments skeptically on the originality and voice in the resulting articles, highlighting a potential downside of relying on LLMs. An actionable insight is to balance AI assistance with personal voice to maintain authenticity while benefiting from structural improvements.

The discussion centers on attempts to integrate large language models like ChatGPT into finance-related tasks and spreadsheet usage. One user reports significant issues with hallucinations and lack of reliable feedback mechanisms, highlighting risks of silent errors in financial systems. Another user finds practical value in using LLMs for generating data connections and queries in spreadsheets, which simplifies data import tasks despite not involving complex formulas. The insights suggest cautious use of LLMs for critical finance operations while acknowledging their utility in automating mundane spreadsheet tasks.

The thread discusses a user's dissatisfaction with Anthropic's customer service despite their rapid growth and promising projects. Another user provides insights into the company's business model, explaining that current subscribers may not cover operational costs, suggesting investors as the primary customers. The discussion highlights technical and cost challenges affecting service quality and pricing, offering alternative solutions like using other APIs for cost efficiency.

The original poster shares an experience where large language model (LLM) agents often fail to follow their own plans reliably. They suggest a solution of managing execution like a todo list, tracking every step explicitly, limiting tool usage to planned steps, and verifying outcomes rather than trusting the agent's claims. This approach, alongside guardrails and version control, significantly improves agent reliability. A respondent agrees and indicates intent to adopt a similar strategy in their own agent development.

Two users share contrasting experiences with LLM-generated emails: one criticizes overly verbose and scope-expanding AI-generated messages from a vendor, advising simplified bullet points instead; the other embraces such emails from a partner regardless of errors. The insight is that LLMs in communication require clear guidelines tailored to relationship and project context to avoid confusion and ensure effectiveness.

The thread shares personal experiences with LLMs applied to spreadsheets. One user highlights the potential of LLMs to identify inconsistent or erroneous formulas in complex, collaboratively edited spreadsheets, rather than just generate new formulas. Another user recounts a positive interaction where an LLM (Copilot) successfully inferred a pattern in manual text edits within an Excel sheet and suggested accurate automated changes, noting that such practical assistance is more valuable than LLMs generating incorrect or irrelevant outputs. The actionable insight is to focus on leveraging LLMs for enhancing accuracy, pattern detection, and error checking in spreadsheet workflows instead of relying solely on formula generation.

The original poster shares a personal experience using GPT-5 for coding, noting that while generative AI can handle many sources and tasks, it still makes elementary errors and lacks in-depth reasoning, leaving the joy of coding intact. Another commenter highlights variability in GPT-5 models, suggesting outcomes depend on the specific model variant used. This thread offers insight into current limitations of AI coding assistants, emphasizing human coding remains valuable for complex problem-solving and fulfillment.

The thread features a positive reception of a blog post detailing a personal journey in understanding large language models (LLMs). The author shares their frustrations and invites feedback on their project 'create-llm'. This encourages community engagement and collaborative improvement of LLM tools.

The discussion emphasizes the practical experience of using local LLMs for programming, highlighting how users often work through bugs by iteratively editing generated code without clear separations between AI and manual input. It suggests that organizations like the FSF should involve practitioners with real hands-on experience to fully understand LLM workflows rather than relying on theoretical notions about prompt and code generation separations.

The discussion focuses on adapting puzzle design with the influence of AI, specifically large language models (LLMs). The original post reflects on the value of maintaining the essence of Advent of Code puzzles despite changes, emphasizing appreciation for past and upcoming challenges. The response highlights the challenge of creating puzzles solvable by humans but resistant to AI solutions, sharing an example based on message cracking. This suggests a community interest in evolving puzzle design to preserve human engagement while acknowledging AI capabilities.

A user shares positive experience using an LLM-based tool for generating diagrams and highlights the productivity gains from being able to interactively refine layouts. The project maintainer responds appreciatively.

A user shared an experience implementing dynamic bug fixes in Ruby metaprogramming by leveraging LLMs to auto-correct undefined function calls during tests. Another participant expressed interest, indicating curiosity about the approach. This highlights a practical application of LLMs in dynamic code repair, suggesting potential for further exploration and sharing implementation details to inspire adoption or improvement.