Python: Making ctypes Structures Beautiful (opens in new tab)

(y4kstudios.com)

44 pointsthey4kman13y ago8 comments

8 comments

People never seem to fully understand namespaces. You don't need any additional prefixes for classes and functions inside of your module. Its name already does separate it from everything else. Moreover "from module import *" is evil. Period.

import newstruct as ns

...

x = ns.long()

DeepDuh13y ago

Makes me think.. is the NS prefix in cocoa frameworks short for "newstruct"? I always wondered about that one.

spleeyah13y ago

    > Cocoa classes begin with the acronym "NS" (standing either for the
    > NeXT-Sun creation of OpenStep, or for the original proprietary term
    > for the OpenStep framework, NeXTSTEP)

http://en.wikipedia.org/wiki/Cocoa_(API)#Cocoa_history

jessedhillon13y ago

That's my module. Now, how can I make it better?

You could have those fields know their own sizes, and then you could have Struct subclasses know the size of their instances, so you don't have to call sizeof. E.g.:

    class DataHeader(Struct):
        address = newstruct.long()
        typ =     newstruct.byte()
        tag =     newstruct.short()
        data =    newstruct.byte(length=40) # 40 bytes

    >>> fp = open('mydata', 'rb')
    >>> dh = DataHeader.read(fp)
    # or perhaps: DataHeader.load(fp.read(len(dh)))
    # this way the side-effect of moving fp's current position is more explicit

    >>> fp.close()
    >>> len(dh)
    12345    # or whatever, sum of sizes for long, 41 bytes, and short

JoachimSchipper13y ago

Additional issues: is a long guaranteed to be 64 bits, or whatever the platform calls long? Is the structure padded (in the same way that the C compiler pads structs)? And, of course, which endianness is used?

jevinskie13y ago

My go-to library for binary mucking in Python is the fantastic Construct library. [0] [1] It is a declarative way of specifying binary structures. It handles endianness, serialization, and deserialization for you. You can build structures of structures with Construct. I used it to parse the PS3's pseudo-ELF files (SELFs) and was quite successful except for a few shortcomings in the library. I don't blame Construct though because the SELF format is a wild beast. Corbin Simpson (the author) was quite helpful when I was looking into the possibility of fixing my issues. Look at the ELF parser to see how easy it is to use! [2]

[0]: http://construct.readthedocs.org/en/latest/

[1]: https://github.com/construct/construct

[2]: https://github.com/construct/construct/blob/master/construct...

wisesage500113y ago

One thing I really like about the struct module's format string is the ability to easily specify the endian-ness of the data. How does newstruct (or the underlying ctypes) handle endian-ness (if at all)?

veyron13y ago

endianness only matters when you bother interpreting the data. The storage concept shouldn't care -- after all, 8 bytes is 8 bytes is 8 bytes.

Struct needs it because it is interpreting data. unpack needs to know the endianness to correctly interpret numeric types, and pack needs to know the endianness to know how to serialize.

FYI: the C way of doing this is byteswapping.

j / k navigate · click thread line to collapse