AI-Generated Cases

One of the ultimate goals for this project was to try and generate a new Ace Attorney 'case' based on all the text of the series. The main inspiration for this was fans' of the series unique creations. If you go to YouTube and search 'AI generated Ace Attorney', you'll find a bunch of videos where people get AI generated dialogue from somewhere like AI Dungeon and visualize it with Objection.lol, an online visualization tool often used by members of the fandom to create memes.
They usually go like this (youtube link):

I think a large reason creations like theses are so popular is that it helps fans appreciate the original source material more and more as it grows older. It's hilarious to see the random stuff the AI comes up with, but Ace Attorney isn't really like that. The series originated as a cheaply made visual novel with a skeleton crew, but it's tight writing, well-written characters, and (usually) sensible and unique mysteries led it to become one of Capcom's longest-running franchises as well as one of the most succesful visual novels of all time.
We wanted to recreate this absurd dichotomy by using our text data to train an AI model and have it create a new Ace Attorney case, hopefully as entertaining as others' creations. Fundamentally though, we had to lower our expectations.

This page is not going to go into extreme detail regarding how text generation works, but basically, an AI model is trained on certain data, and the actual generation is just the model predicting the most-likely next word, or 'token'. ChatGPT, the most famous AI model of today, is just a hyper-advanced version of this typical text-generation model. That's why you'll hear about Open-AI getting into trouble for stealing data, their model requires TONS of data to be so intelligent!
The point being, once we train a model on our Ace Attorney text, we probably won't be able to make anything as new or creative as those AI Dungeon videos, since the model would only be picking text from it's vocabulary (full of tokens from the text itself).

Obstacles aside, let's train a model! We narrowed in on this TensorFlow tutorial since the example they used generated text based on Shakespeare's writings, and generated a script-like text similar to what we wanted to make. We rewrote their code to work with our data (check out the python file!), tweaked with the length of the generation and the temperature (which affects how random the text prediction is), and this is what we got!

Wow! That was something!
These results may seem dissapointing, even considering our lowered expectations, but there's a lot of compelling stuff here!
Observe this snippet (or check out the full generation here):

The generation does follow the script-like formula we were hoping for, and certain characters' names appear fluently! Edgeworth crops up several times, though two characters called Gro and PhounHossillo have appeared as well, interpreted as Grossberg and Phoenix for the purposes of our visualization.
I think there are two main reasons why this generation may have failed so spectacularly. For one, the tensorFlow model trains by splitting each word into characters, and the generation is based off predicting the very next character in the sequence. That's why you'll see names like PhounHossillo, the model thinks it's seeing the name Phoenix, writing P-h-o, before diverting into something else, possibly the result of our temperature.
For a more explicit example, look at when the name Kazumshoe appears in the full generation. This is an obvious combination of the two characters named Kazuma and Gumshoe, the model's attention span is flawed.
For a deep look into attention in this context, check out this 3Blue1Brown video!

The main takeaway of this expirement was that training a language model is really hard! There are a bunch of parameters the author can tweak with to adjust how the model is trained, how each token affects the others, how the probability matrix for the next token is handled, and so on.
Since training our own model wasn't very fruitful, we wanted to cheat a little, by using the most popular language model on the market, ChatGPT.
Our biggest question for this avenue in the project was to see if the Ace Attorney text could be self-sustaining, as in, if a model trained only on the text from the franchise be able to generate similar-enough cases, but as we've said, that didn't pan out that well.
That being said, seeing if ChatGPT can generate an Ace Attorney case is another way of answering that question, at least, if you squint. If Ace Attorney is a 'big' enough franchise, then ChatGPT probably has Ace Attorney data somewhere in its training data. So, we tried it out! Here's the visualiazation (or here's the full generation):

To begin the analysis of this generation, there are several things that ChatGPT gets right, small details about the franchise that I didn't expect it to be wise to. For one, the object of attention for the trial is a stolen sword of Khura'in, which is pretty close to an important item from the third game, not to mention the setting of Khura'in being name-dropped (though the model seems to have confused Kurain Village and the country of Khura'in). There are some small details it gets right too, like Gumshoe's preference for calling everyone 'pal'.
However, it gets a whole lot wrong as well. Most obviously, Phoenix Wright is acting as a Prosecutor, which isn't accurate to his character. In the conclusion to the case, a jury decides the ruling, something unheard of in the majority of the mainline games.
All of the court dialogue feels incredibly boring, as well, the series' distinct rhythym and gameplay loop is completely absent.

But putting minute criticisms aside, the biggest flaw with this generation is its complete lack of creativity. The chatbot has taken the 5 most well-known characters from the original 2D games and made them interact in a low-stakes, random courtroom environment with virtually no imagination employed. There's no character-work being done, the way the case resolves is just uninteresting, and perhaps the most remarkable thing of all, the generation is unimaginative enough to not even employ the Ace Attorney universe's rules, it uses a jury system instead. The courtroom scene plays out much closer to a real-life, U.S. courtroom scene than anything from the Ace Attorney franchise.
To wrap this analysis up, it's clear that creating a text-generation bot built to generate Ace Attorney cases is possible, but there's honestly no good reason to. Ace Attorney thrives on creativity and passion, it's the reason why the franchise has such a burgeoning online fanbase even today. To strip away the series' creativity would truly be a shame. The ChatGPT generation is competent enough, but it truly is less imaginative than the series at its worst.