Local AI Integration Project

Author:

🔗 Trustingmeerkat

Real discussions and feedbacks of Local AI Integration Project

🔗 Join the Discussion on Forums

App Description

I’m currently building a system that takes the text content of news articles about cocktail competitions and then attempts to extract a JSON object from it via using phi4 local ai model.

I’m developing it alongside Claude in a project and we’ve built so far a series of qualifying questions that is prompted to phi4 and it’s answered are formatted to JSON

I’m attempting to one shot each answer with the specific question and content of the article by asking the same question to phi4 3 times and picking majority answer.

Then, the flow of questions are conditional so that the ai is provided a set of questions based on previous answers.

I’m getting decent results and anecdotally it’s about 50% correct. So I think I need to begin prompt engineering to get better. Except, I’m wondering if there’s a way to automate these iterations a bit? Currently I’m pasting code and results into 01 preview and asking for detailed analysis, then passing this back into Claude for code revisions all manually.

I guess I should design an accuracy test (again with ai) across 10 or so random articles at a time and a/b test until we get something we’re happy with? Does anyone else have any suggestions?

I also previously attempted to one shot the entire JSON object rather than elect to flow through a bunch of questions except that didn’t work so well and decided to pivot rather than keep trying to optimise it.

Project Overview

The project involves integrating a local AI model, phi4, to extract JSON objects from news articles about cocktail competitions. The process includes prompting phi4 with a series of qualifying questions, formatting answers to JSON, and using conditional flows based on previous answers. The current accuracy is anecdotal at about 50%, with efforts underway to improve through prompt engineering and possibly automating iterations. The author is considering designing an accuracy test across random articles for A/B testing to refine the system.

Features & Benefits

✅ Integration of local AI model for data extraction
✅ Use of conditional questions based on previous answers
✅ Potential for automation and improvement through prompt engineering

Areas for Improvement

🔄 Current accuracy is only about 50%
🔄 Manual process for code and result analysis
🔄 Previous attempt to one shot the entire JSON object was unsuccessful