Decrypt cardinfo.dat?

Siberys · Joined: 31 Mar 2008 Posts: 4

Does anyone know a way to decrypt the cardinfo.dat file, so I can read it in plain text?

Thanks!

Gredunza · Joined: 04 Apr 2008 Posts: 3

I would also dearly love to know this one.

I've read this thread, which contained very little relevant information, aside from a redirection to MakeAppr.
Now, I don't know C/C++, so it helped me very little in discovering the structure of the thing and when I tried to actually use the program, it threw an exception and died on load.

I do, however, know Perl (you may gasp with disgust). And if there's anything Perl can do is process text.
So, I've spent the better part of the afternoon trying to reverse engineer the thing and I did come up with some information.

What I do know, and take it with much skepticism:
There are two concatenated blocks, with no line separators (that's right. NONE), a "Card Info" block, which basically operates like so, per record:
1. A character which I think stands for card color. This is usually a control character in the C0 (\x01-\x1F) range, but not always.
2. The NUL (\x00) character.
3. The card type text.
4. NUL or, if available, the STX (\x02) and NUL characters, followed by the Casting cost.
5. NUL or, if available, the ETX (\x03) and NUL characters, followed by power/toughness in N/M format.
6. A character I've yet to understand.
7. The NUL character.
8. Card text, separated by newlines (\n) (Oh, the twisted web you've weaved).
9. The DC4 (\x14) character and NUL characters.
10. Flavour text (Nowadays "www.Magic-League.com") (which makes an excellent line separator for this block).

It's fairly obvious the NUL char is the tokenizer. But that's all I'm certain in.
While I do get it, I'm not sure how to process it without record delimiters. Probably just regex it, as it seems the most sensible thing to do.

Now, all this is followed by the "Card name / editions" block. Which operates similarly, but I've yet to completely perceive its resonance.

Obviously, people are still adding sets to the file, so if anyone cares to post the full specification, that would extremely nice.

Gredunza · Joined: 04 Apr 2008 Posts: 3

Man, was I naive....
After much trial and error, this is the regex I came up with:

Siberys · Joined: 31 Mar 2008 Posts: 4

You'll have to excuse my ignorance, but I'm not exactly what you'd call proficient in perl; How can I run that program?

Thanks

Gredunza · Joined: 04 Apr 2008 Posts: 3

You shouldn't run that program yet. It's partial and incomplete.
If you are interested in Perl, however, or would like to get even this partial information, you'll need to install Perl.

If you're using Linux, I suspect you have, at least, Perl 5.8 or you know how to install either 5.8 or 5.10.

If you're on Windows, you can install ActivePerl or Strawberry Perl (the only Perl with a giant strawberry!). Both are free, of course.
Once you're done with that, it should be as simple as saving all that code in a file, say 'cardinfo.pl', opening cmd, going to your Apprentice\Sets\ directory and typing "perl P:\ath\to\cardinfo.pl cardinfo.dat > cardinfo.txt"

Now, I'm working with a friend on writing up the whole specification for the file.

Which, with all due respect, is one of the worst and strangest databasing schemes I've ever seen. A CSV would've been better.

The first block can realistically be parsed with a Regular Expression, but it's potentially incorrect and, IMHO, just ugly as it would require quite a bit of hard coding. I try not to hard code.
This is because the first block, containing card type, casting cost, P/T, card text and flavour text is one continuous stream of characters.
I earlier said that there are no record delimiters, this was only partly correct, as there are no delimiters at all.
It's a bit confusing, but, basically, it has the length of a field, followed by the field text, but neither is a constant.
The second block is far more civilized, with each record ending in two NULs.
Anyhoo, we hope to have this straightened up soon.

Siberys · Joined: 31 Mar 2008 Posts: 4

Thanks! Good luck on the project - you don't know how useful it'll be to me.

Siberys

Brefin · Joined: 31 Oct 2007 Posts: 18

Since you seem to have an interest in Perl and such. I thought I'd pass on something helpful.

Attached is a rough script for reading cardinfo.dat. The script works, but as you can see I was lame in giving the script the file to work with.

I used what I learned in making this script to adapt a gatherer reading script to output cardinfo.dat. A key thing is to treat cardinfo.dat as a binary file and use perl's packing routines to convert the values as needed to read and write the file.

Brefin · Joined: 31 Oct 2007 Posts: 18

I see... the original reference I used has been removed from the web. But I located the page in the wayback machine: http://web.archive.org/web/20050503155724/http://www.digitalsquirrel.com/warlord/appr2txt.html

The database format isn't dumb, just the binary output for a language that speaks more of binary structures in memory. I've seen worse, for instance, outputting game data as straight memory dumps of what's in memory. Talk about hard to move between platforms, but it loads plenty fast on its original platform without extra work.

Edited: Fixed broken URL. Always remember to preview!