Code

This page contains information about all the data we gathered when making this project. You'll be able to look at how we obtained our data, as well as samples of the resulting XML code we created with RegEx expressions.
Basically, our workflow for data-gathering went like this:
Online transcripts -> Text files -> XML files
Then we could use that structured data to analyze trends in the series like characters' numbers of lines. For more about that, check out the Analysis Page.

To begin with, we needed to decide on a source to glean our data from. Obviously we couldn't play all the games while copying each line! The source we settled on was the Ace Attorney franchise's Fandom page. A Fandom page is an online encyclopedia page about a movie, TV show, or game series, created and moderated entirely by fans. The Fandom site has transcripts of all of the chapters of the series. However, the HTML behind these transcripts was very intimidating to read, as you can see from this brief sample:

Not pretty... Spoiler alert; there are over 2000 lines of code that we don't care about before the links to the individual transcripts appear. Once we've locked down where exactly the links we care about appear (in this case, their found in the div element with an id of "mw-parser-output"), I was able to download the contents of each transcript page using python code, viewable here (or here, edited from a newtfire file):

We knew from the outset that we wanted plain text files we could train a text-generation model on, and XML files we could analyze and glean data from. We could do this in two stages; First, make our text corpus, then create our XML corpus from the text files (creating XML files from the raw HTML transcript files proved to be far too complex).

The process of turning the HTML transcripts into text files was pretty simple, it mostly involved identifying the part of the page where the in-game text appeared (it was in the same section as the transcript links, the div element with the "mw-parser-output" id), and removing all the HTML remnants that didn't represent anything from the game. Importantly, some of the most iconic lines in the game, those being the 'Objection!'s and 'Take That's, and other lines of such nature, weren't represented by any text, rather they're represented by PNG's or GIF's, like the one seen to the right. To solve this, we carefully carved out each one of these special cut-in speech bubbles with precise XQuery commands going over the entire collection of HTML transcripts, with our future purge of the HTML remnants in mind. For a full record of the searches/replaces made in the process, you can read our markdown file that recorded them!

As for the process of turning these text files into XML files, the process was a bit buggy, but by identifying how the scripts were structured, we could wrap almost every line in its own element that identified the speaker in an attribute (Check out the markdown file cataloguing this here. The resulting files contained several bugs, but they were pretty simple to fix, we just had to go through the files one by one to see what errors were popping up, then fixing them with XQuery (The markdown for this process is here). It's a good thing the errors were as uniform as the code!
Here's a snippet of the resulting XML code:

The data we've scraped up isn't perfect, there's remnants of the HTML transcripts' walkthrough-like structure, as well as some lines with speakers that aren't real characters, but the resulting corpusses (corpi?) are comprehensive and all well-formed, a mighty feat considering how buggy certain steps in the process were!
Here's a series of download links to all our data corpusses, as well as the combined files containing all the data:

• Text Corpus (zip folder)
• XML Corpus (zip folder)
• Combined text file
• Combined XML file