There are already bots that use something like 5 specialist bots and have them sort of vote on the response to generate a single, better output.

The excessive prompting is a necessity to override the strong bias towards certain kinds of results. I wrote a dungeon master AI for Discord (currently private and in development with no immediate plans to change that) and we use prompts very much like this one because OpenAI really doesn't want to describe the actions of evil characters, nor does it want to describe violence.

It's prohibitively expensive to create a custom AI, but these prompts can be written and refined by a single person over a few hours.

It's already been done, for at least a year. ChatGPT plugins are the "different frameworks", and running a set of LLMs self-reflecting on a train of thought, is AutoGPT.

It's like:

1. Can I stick my fingers in a socket? - Yes.
2. What would be the consequences? - Bad.
3. Do I want these consequences? - Probably not
4. Should I stick my fingers in a socket? - No

However... people like to cheap out, take shortcuts and run an LLM with a single prompt and a single iteration... which leaves you with "Yes" as an answer, then shit happens.

You are using the LLM to check it's own response here. The point is that the second LLM would have hard-coded "instructions", and not take instructions from the user provided input.

In fact, the second LLM does not need to be instruction fine-tuned at all. You can jzst fine-tune it specifically for the tssk of answering that specific question.

Yeah, as soon as you feed the user input into the 2nd one, you've created the potential to jailbreak it as well. You could possibly even convince the 2nd one to jailbreak the first one for you, or If it has also seen the instructions to the first one, you just need to jailbreak the first.

This is all so hypothetical, and probabilistic, and hyper-applicable to today's LLMs that I'd just want to try it. But I do think it's possible, given the paper mentioned up at the top of this thread.

Any input to the 2nd LLM is a prompt, so if it sees the user input, then it affects the probabilities of the output.

There's no such thing as "training an AI to follow instructions". The output is just a probibalistic function of the input. This is why a jailbreak is always possible, the probability of getting it to output something that was given as input is never 0.

Ah, TIL about instruction fine-tuning. Thanks, interesting thread.

Still, as I understand it, if the model has seen an input, then it always has a non-zero chance of reproducing it in the output.

Because it's probibalistic and in this example the user's input has been specifically crafted as the best possible jailbreak to get the output we want.

Unless we have actually appended a non-LLM filter at the end to only allow yes/no through, the possibility for it to output something other than yes/no, even though it was explicitly instructed to, is always there. Just like how in the Gab example it was told in many different ways to never repeat the instructions, it still did.

I'm confused. How does the input for LLM 1 jailbreak LLM 2 when LLM 2 does mot follow instructions in the input?

The Gab bot is trained to follow instructions, and it did. It's not surprising. No prompt can make it unlearn how to follow instructions.

It would be surprising if a LLM that does not even know how to follow instructions (because it was never trained on that task at all) would suddenly spontaneously learn how to do it. A "yes/no" wouldn't even know that it can answer anything else. There is literally a 0% probability for the letter "a" being in the answer, because never once did it appear in the outputs in the training data.

Oh I see, you're saying the training set is exclusively with yes/no answers. That's called a classifier, not an LLM. But yeah, you might be able to make a reasonable "does this input and this output create a jailbreak for this set of instructions" classifier.

Edit: found this interesting relevant article

Mastering LLMs for Complex Classification Tasks - Olaf Lenzmann - Medium

Checking if some text contains an answer to a question — how good are state-of-the-art LLMs at doing that? We put GPT and Claude, Gemini, PaLM 2, Mixtral, and Cohere through their paces. We evaluate…

^{Olaf Lenzmann (Medium)}

Blog commenter Frank Wilhoit made a now somewhat famous assertion that the human default for nearly all of history has been conservatism, which he defined as follows:

There must be in-groups whom the law protects but does not bind, alongside out-groups whom the law binds but does not protect.

He then defined anti-conservatism as opposition to this way of thinking, so that would be to ensure the neutrality of the law and the equality of all peoples, races, and nationalities, which certainly sounds left-wing in our current culture. It would demand that a legal system which protects the powerful (in-groups) while punishing the marginalized (out-groups), or systematically burdens some groups more than others, be corrected or abolished.

The problem with a "beneficial to humanity" axis is that I think that most people think their political beliefs, if enacted, would be beneficial to humanity. Most people aren't the villains of their own stories.

The very act of politics is to disagree on what is best for humanity.

If you think about it logically, there are some core things that are always good. Like considering everyone to be inherently equal. While there are things that muddle even this point, it still wont take away that you should always keep those core principles in mind. Religious teachings have pretty good point about this with "treat others like you want yourself be treated" and "love even your enemys". That is the only logical way to do things because to do otherwise leads to all of us either just killing each other or making life miserable so we want die.

I had some other thought about this too, but i cant seem to be able to properly put it to words at the moment. But the idea was that we should all try to think about things without ego getting in the way and to never lie to oneself about anything or atleast admit to ourselves when we have to do so. The part i cant seem put to words is the part that ties to the previous thing i said.

You are right.

But I try to think things as objectively as possible and hope others would too (but dont expect it).

No one probably thinks what they are doing is wrong or at least try to find justification for it, objectively there are things that cause good or bad things regardless of your intentions. While good results don't excuse evil actions, bad results are still bad results regardless of your intentions. Its ok to try even if there are risks, but one should always consider if the risks outweigh the results. And sometimes even if everything goes according to plan, it might still cause things to happen you end up regretting and it would have been better for everyone if you had thought it more.

That is what i wish people thought more instead of limiting themselves to just political things and easy terms. Ultimately it doesn't matter who is in power but what it causes.

This is an economic model known most commonly as feudalism.

Hahaha, that's not how feudalism works at all. You are twisting yourself backwards through your legs to come up with some kind of nonsense that makes Stalin not far-left. It's hilarious.