Ayai · GPT Nexus - ChatGPT, Claude, Gemini, Perplexity, Local LLMs

[zeitlings]

Hey everyone!

 

I'm excited to share the alpha preview of Ayai - GPT Nexus with you, a workflow that lets you interact with various AI language models.

The workflow integrates APIs from OpenAI, Anthropic, and Google, while also supporting local LLMs.

 

Why an Alpha Preview?

 

While Ayai - GPT Nexus is still a work in progress, it’s already quite useful. Development has slowed down recently, and I’m not actively working on it as much. Rather than keep it under wraps until it's perfect, I wanted to release it now so you can take advantage of its current features and provide feedback that could shape its future development.

 

Current Features

Chat with AI Models: Interact with ChatGPT, Claude, Perplexity, and Gemini. Third-Party Proxies: Connect through OpenRouter, Groq, Fireworks, Together.ai, and other proxies compatible with OpenAI's response format. Local LLMs: Use local models via interfaces like Ollama or LM Studio. Live Web Search: Enable live web search for OpenAI models with additional Exa.ai or Azure API keys (experimental).

 

What’s Missing?

 

The "Ayai" part of the name hints at some personal AI assistant features I have in mind and plan to explore in the future. Additionally, the text processing features like paraphrasing, simplifying, changing the tone of text, or summarizing documents (PDFs, docx, text files, etc.) without needing to go through the chat window are still on the to-do list.

 

Your feedback is, of course, appreciated!

 

Download here: Ayai · GPT Nexus on Github

 

 

 

Looking forward to hearing what you think!

 

Edited June 12 by zeitlings

[zeitlings]

I want to call attention to some aspects that are easily overlooked and share some tips.

 

Note the hidden options, i.e. modifiers mentioned in the documentation to make use of 

• Multi-Line prompt editing
• Vocalization of the most recent answer (See here how to change the voice)
• The HUD to quickly check your most important current settings.

Some third party proxies allow you to use certain models for free.

• For example, Groq currently grants access to Llama 3 70B, which is en par with Gemini 1.5 Pro according to some benchmarks.
• OpenRouter curates a list of free models that can be used with their API.
• The new API for Google's Gemini is currently in beta. If your API requests are detected as coming from the US, Gemini comes with a substantial free tier quota for both Gemini 1.5 Flash and Pro.

I also encourage you to explore local LLMs. Meta recently released their Llama 3 model, and the "tiny" Llama 3 8B performs beautifully on newer Macs. According to some benchmarks, the 8B version competes with GPT 3.5 Turbo, and I can confirm that I was at least very positively surprised by its capabilities. Related: Ollama Workflow.

 

 

[rudraadavee]

The continue with existing chat action leads to nothing in the workflow editor. Overall pretty nice work.

[zeitlings]

sigh thanks. Fixed.

[rudraadavee]

hey messed with the workflow for a while and found that the "new chat" function has stopped working and also could you help me with the path in "openai_alternative_local_url_path" in case of local LLMs? thanks

[zeitlings]

6 hours ago, rudraadavee said:

hey messed with the workflow for a while and found that the "new chat" function has stopped working

 

I've just checked all the possible ways to start a new chat, and they all work on my end. (Without more information, I can't infer where it stopped working for you or why.)

Don't forget to keep an eye on the debugger to potentially identify any problems more easily.

 

6 hours ago, rudraadavee said:

also could you help me with the path in "openai_alternative_local_url_path" in case of local LLMs?

 

Sure, what's the problem? 

Edited May 31 by zeitlings

[dood]

Hi @zeitlings, I'm curious about the integration with Exa.ai. If I add an API key, does every message I send through the workflow get filtered through Exa.ai? Or is only certain messages based on how the prompt is formulated? Thanks!

[zeitlings]

Only messages that the model deems to require real-time information will elicit a detour through exa.ai. OpenAI dubbed this tool- or function calling.

I caution the model to be extremely conservative about calling the function, so ideally it should only happen when really necessary. Whenever it does happen, you will notice that a "Searching" notification is injected into the conversation! This is when the external API is contacted, otherwise exa sees nothing, and even then it doesn't see your message, but only a search query constructed by the model to get the information it needs.

[dood]

The Exa.ai integration works great! I'm unfortunately unable to start a new chat, though. The following error appears in the debug log:

 

[16:11:00.309] STDERR: Ayai · GPT Nexus[Run Script] mkdir: /Users/*/Library/Application Support/Alfred/Workflow Data/com.zeitlings.gpt.nexus: No such file or directory
mv: rename /Users/*/Library/Caches/com.runningwithcrayons.Alfred/Workflow Data/com.zeitlings.gpt.nexus/chat.json to /Users/*/Library/Application Support/Alfred/Workflow Data/com.zeitlings.gpt.nexus/archive/20240606-1611-b910df14f9c640c983679ff663ed008e.json: No such file or directory

 

[zeitlings]

Thanks for the error message.

 

For a quick fix, you can try replacing this

[[ -d "$archive" ]] || mkdir "$archive"

with this

[[ -d "$archive" ]] || mkdir -p "$archive"

here

[iandol]

Wow, amazing set of supported interfaces, thank you @zeitlings -- beautiful icon too!!!

 

On github, alpha.1 is newer than alpha.2, I assume alpha.2 is the better one to download though?

 

[zeitlings]

3 hours ago, iandol said:

Wow, amazing set of supported interfaces, thank you @zeitlings -- beautiful icon too!!!

 

Thanks 🤗

 

3 hours ago, iandol said:

On github, alpha.1 is newer than alpha.2, I assume alpha.2 is the better one to download though?

 

Is it? I noticed that the sorting looks wrong when viewing all releases, do you mean this? But alpha.2 should be set as the latest release. So yes, alpha.2 has some fixes and changes over alpha.1. However, mind that the problem with the data directory not being created is not fixed yet.

[iandol]

Fixed and it is working well with LM Studio!!!

[zeitlings]

sneak peak 👀

 

[zeitlings]

Added Claude 3.5 Sonnet

[philocalyst]

Love the utility! Thank you for putting this together. I use a dark theme, and I like larger text in general, and I don't see options to edit the font size or colors of the interface... Is this something that could be implemented? 

[zeitlings]

Hey @philocalyst, thanks! Glad to hear it.

 

At the bottom of the configuration you can find a section called "Accessibility" where the text size can be changed. 

 

The available sizes are enforced by Alfred's text view and cannot be customized further. 

 

Generally, the look of the text view is determined by your current theme, i.e. changes in color or font size there will be reflected in the Ayai chat window. 

Unfortunately, ad hoc changes to the theme of a specific object (such as the text view used for the chat window), without affecting the theme everywhere else, is currently not possible.

[zeitlings]

Ayai GPT Nexus alpha.5

• Added Inference Actions 🥳
  • Requires accessibility access
  • Inference Actions are evaluated with pkl and can be freely modified or extended
• Added configuration option: Generate
• Added Environment Variable openai_alternative_shared_finish_reasons to handle additional stop sequences #2
• Updated models
• Updated annotated chat service images
• Updated workflow documentation

 

Inference Actions provide a suite of language tools for text generation and transformation. These tools enable summarization, clarification, concise writing, and tone adjustment for selected text. They can also correct spelling, expand and paraphrase text, follow instructions, answer questions, and improve text in other ways.

 

The results will be either streamed, pasted or copied to the clipboard. The default behaviour can be set via the Generate configuration option or selected en passant via the key modifier combinations. Ayai will make sure that the frontmost application accepts text input before streaming or pasting anything, and will simply copy the result to the clipboard if it does not. This requires accessibility access, which you may need to grant in order to use inference actions.

 

• ↩ Generate the result using the configured default strategy.
• ⌘↩ Paste the result and replace selection.
• ⌥↩ Stream the result and preserve selection.
• ⌃↩ Copy the result to clipboard.

 

[philocalyst]

Just a general concern, which caused another problem as I tried to fix it, for some reason when I send in a query through the workflow it will somehow use 40,000+ tokens?! Then I tried to shorten my context window (even though I'm not within the actual chat view so it shouldn't matter) and now it's saying 

 

messages: first message must use the "user" role

 

Is this expected behavior? Don't know what to do, I find it very convenient just not worth the 20 cents per query.

[zeitlings]

Hey @philocalyst

 

Keep in mind: 

Quote

If you opt in to using more expensive models, be aware of their prices and consider starting new conversations often to avoid paying for unnecessarily large chat histories included in the context window.

 

If you enable live-search (and use ChatGPT) there will be an additional ~500 tokens used per request. The system prompt also comes on top each time. 

 

You can also limit the count of messages that are included in the prompt via the configuration, effectively shortening the context window:

 

4 hours ago, philocalyst said:

messages: first message must use the "user" role

 

How exactly did you try to shorten the context window? If you tried to shorten it via the configuration, and the chat history contains a tool call ("Searching"), I have a hunch that the truncation could be the problem. In that case, try changing the range a bit for now (+1 or +2 or +3, -1 or -2 ...). If that solves the problem, then the way how the context is truncated might actually need to be refined.

 

[philocalyst]

Got it. So my history was growing exponentially and that was what was reflected in the tokens. For some reason I thought if I wasn't in the chat window it was a new conversation each time. The context option is still misleading to me, I wish it was clearer those are messages included, it was my first assumption but I feel it's fair to say how "context" is dealt with in AI engineering is by tokens not chat messages. Thank you for being so kindly dealing with these issues, it's generous of you.

[zlebandit]

Hi all. I use this wonderfull Workflow but since this morning, I've got this error : [File System Error] Chat file does not exist.

I don't know how to manage it ; can someone help ?

[zeitlings]

Hey @zlebandit

I'd be interested in the sequence of events that led you to this scenario, if you recall. 

Which version of the Workflow are you using? 

 

As a first aid measure, try starting a new conversation or canceling response generation (this will clean up some of the control-critical files that may be lingering due to some unknown exception, but shouldn't).

 

Edited August 30 by zeitlings

[zlebandit]

Hello and thank you for your answer. 

I use de Alpha 5 version and I was doing an integration in a new workflow in order to make an "action menu" for Universal Action with prompts that I often need.

[zeitlings]

Hey @zlebandit

I see. Make sure you use the External Trigger object with the identifier "chat.new" when you are sending the argument (prompt) and variables to the Nexus via an external workflow! That way your previous conversations will be preserved and the chat file won't be missing.