Home > Dev, Featured > Random Accessing Zip Files with Adobe AIR

Random Accessing Zip Files with Adobe AIR

September 20th, 2010 Leave a comment Go to comments

I’ve recently re-published a new version of hexagonlib, a universal AS3 class library at code.google.com/p/hexagonlib/. Some parts that were originally in the library have been removed, in particular the UI components and the game package. This has been done because I’m working on a game engine (more about that one later) that will probably exclusively include these parts. The hexagonlib is instead targeted at a broader area of development, not just games.

Either way, many classes have been updated and improved (and many still need too *ugh*) and what is particularly worth mentioning are the file IO classes which provide a unified way to work with different file formats. Basically the way how files work in hexagonlib is that you can create file objects of any specific file type (like text, binary, image, XML, etc.), give them a path to a physical file and then add them to a loader (BulkLoader, FileLoader, ZipLoader) which then loads the data of the physical files into the file objects.

While you can use the BulkLoader class to load a collection of arbitrary files in one go with all sorts of comfort (priorities, weighted loading, load retries, multi-connections etc.) the newest addition to the library is the ZipLoader class which can be used in AIR development to access a standard zip file using Random Access. What does that mean? It means that you can create a zip file (a very large one if you want), pack all your resource files that can be loaded by your application and then open it with your app and ‘load’ (= extract) files from it without ever needing to load the whole zip file completely into memory. This makes accessing a large zip file very efficient because only the chunk of the requested, zipped file is loaded.

This is especially interesting for us game developers who desire to use large, nicely packed resource files like they are utilized in a similar fashion in most current day commercial games. For a while I was promoting to add such functionality to AIR over at Adobe Labs but that was before I knew that this can actually be done in AIR since 1.0 thanks to the FileStream class and the position property of it (alas, the property is not available in the URLStream class so random access is not possible on web-based Flash). The ZipLoader uses asynchronous loading to open a zip file as well as ‘loading’ files from it because I don’t like the idea of having the application at the mercy of the file system which would be the case with synchronous access (and which is used in way too many examples on the web).

You can download the hexagonlib distribution over at Google Code, which includes the SWCs and documentation and of course the source code is available for access via SVN. I’m updating the library on a irregular basis. There are already some Wiki pages too with code examples showing how to use the BulkLoader and ZipLoader here but I will hopefully get to write some more in-depth tutorials soon.

hexagonlib at Google Code
File API Wiki
Documentaion

  1. Ryan
    October 4th, 2010 at 19:07 | #1

    Hi sascha,

    I’m trying to implement ZipLoader in a small app I’m making and although it’s been easy to implement I’m running into an issue when I choose ‘store’ as my ZIP option.

    It looks like Ziploader supports the store method however I receive a few obscure characters (PK) which breaks my string to XML conversion within XMLFile.as

    Have you a workaround?

    Thanks,
    Ryan

  2. October 4th, 2010 at 20:31 | #2

    hi ryan, can you send me the zip file (email is on contact page)? otherwise i have no idea atm. you’re trying to load a zip, not write? also make sure that the packed filenames are all standard ascii.

  3. Ryan
    October 4th, 2010 at 22:31 | #3

    Thanks sascha,

    I’ve sent you an E-mail, narrowed it down to filesize error.

    Hope you can help,
    Ryan

  4. October 19th, 2010 at 17:53 | #4

    Any hints as to how to do this with an encrypted zip? i.e. it would be great to say store the games serial number somewhere secretish on the system (windows registry for example) and then use that to decrypt the zip and load assets…

    Also interested via AIR :)

  5. October 20th, 2010 at 12:43 | #5

    @David Komer
    The ZipLoader (and neither the ZipFile) supports encrypted zip files yet. This would be a feature that needs to be added.

    I’m sure there are good reasons to encrypt a game’s resources, in particular if you don’t want players to tamper with your game but storing the serial number in the Windows registry isn’t such a good idea IMHO. What would you do on Mac and Linux?

  6. October 20th, 2010 at 12:46 | #6

    Hmmm any other idea where to store the serial then?

  7. October 20th, 2010 at 13:17 | #7

    I could think of some way to store it in a file or local shared object and one-way-encrypt it with SHA or MD5 but still somewhere in your app you would probably need a key to decrypt that so it’s never 100% safe. As far as I know the only really safe way would be to use a server to that your app/game connects to check serials. But I’m completely against software protections that require an internet-connection so I don’t want to promote them.

  8. October 20th, 2010 at 14:25 | #8

    Thanks :)

    Can I pass a ByteArray to ZipLoader? That way I could encrypt the file, load it, decrypt it to bytearray, and pass that to ZipLoader…

  9. Ryan
    October 20th, 2010 at 15:59 | #9

    Hi Sascha, I sent you an Email late last week regarding an issue I was having – have you had chance to examine it?

    Thanks
    -Ryan

  10. October 20th, 2010 at 20:28 | #10

    @David Komer
    It doesn’t work that way. In an encrypted zip the packed files would have to be encrypted one by one so that you still can fetch them from the zip independently otherwise it would have to load and decrypt the whole zip at once which isn’t what the ZipLoader was made for. You could however encrypt some files, pack them into your zip file and then extract these with ZipLoader for example by loading them as a BinaryFile from it. Then you get these files as a bytearray which contains your originally encrypted data.

  11. October 20th, 2010 at 20:30 | #11

    @Ryan
    Hey Ryan, yes I got your email. Sorry this takes so long for me to get back to your problem! I will look at it asap!

  12. October 21st, 2010 at 17:31 | #12

    OK- and another question, when you say that ZipLoader doesn’t need to load the whole thing into memory- does that mean if say I have a 1gb zip on a DVD, and I need to grab some small asset for it, the disc won’t just spin around forever loading the whole gig?

    That would be awesome!!

  13. October 21st, 2010 at 17:59 | #13

    @David Komer
    That’s exactly the point of the ZipLoader class and what random access is about! It only loads the chunk of the file into memory that you request. If it had to load the whole gig into ram anytime you access the zip file just to extract a small file from it that wouldn’t be very efficient, would it?!
    That said, don’t take my word for granted that everything works flawless right out of the box when trying to access a 1GB file from a DVD as I haven’t tested it yet with DVDs or CDs, only with harddisks. But theoretically it should work the same.

  14. October 21st, 2010 at 18:06 | #14

    Awesome, thanks :)

  15. Ryan
    October 22nd, 2010 at 21:56 | #15

    @sascha
    Cheers Sascha, I will look forward to your reply!

  16. Krzysztof Jez
    November 14th, 2010 at 04:27 | #16

    Hi Sasha
    Im very interesting in your hexagonlib framework – can I find some kind of working demo on the web? Before compiling I prefer to check its eficiency and so on. I will be gratefull for help.
    Krzysztof

  1. December 18th, 2011 at 14:09 | #1
You must be logged in to post a comment.