denis_dolya

I want to present a methodology I have been working on called SRAW Simple Raw. This is not a conventional compression algorithm but a preprocessing approach aimed at optimizing data before it is formatted or encoded. The key insight is that most data formats carry significant overhead in metadata headers and inefficient encoding that can be eliminated with minimal processing.

With SRAW repetitive patterns can be encoded extremely efficiently for example 1000 identical values can be represented in just a few bytes. CPU overhead is minimal making it suitable for microcontrollers and memory constrained devices. Output sizes are predictable which is critical for embedded environments. The approach can be used alongside traditional compression rather than as a replacement.

Implementation relies on smart bit packing and pattern recognition tailored to specific data types. In practice this has produced significant improvements with sensor streams telemetry and binary protocols where traditional compression either underperforms or requires excessive resources.

The underlying principle is straightforward but often overlooked optimizing data at the source can be more effective than compressing it afterwards. Removing redundancy early typically yields better efficiency than applying compression only after formatting.

I am sharing this to engage with engineers working in embedded systems IoT or other fields where data efficiency is critical. Feedback alternative approaches or similar experiences would be highly appreciated.

GitHub: https://github.com/DenisDolya/sraw

Documentation: https://denisdolya.github.io/sraw/

I am working on a new embedded graphics format that aims to be simple to parse, memory-efficient, and easily implemented on constrained devices without external libraries. The motivation for this project comes from years of experience working with existing formats. PNG is difficult to parse on embedded platforms without significant library support, and BMP, while simple, consumes too much memory for many constrained systems. I wanted to explore a solution that combines the simplicity of BMP with improved efficiency and a structure that is easy to implement directly in code.

I have considered raw-based formats with minimal headers, similar in some ways to PCX, but I want to explore alternatives that remain practical for long-term maintenance and extensibility.

What are the most important considerations you take into account when designing a custom data format for embedded systems that will likely need to be maintained and extended over the next 10–15 years? Specifically, I am interested in trade-offs between simplicity, memory efficiency, and potential future flexibility. Any experiences, lessons learned, or patterns to avoid would be very valuable.