Click here to Skip to main content
15,938,328 members
Please Sign up or sign in to vote.
3.50/5 (2 votes)
See more:
Hello guys
I need to compress data in a byte array and store in database. speed is favored.
I want to compress better than very simple algorythms such as counting repeated bytes.
I want simpler algorythms than lempel-ziv, to compress small count of bytes, say 500 bytes.
Specifically I Want this: 1- compress small arrays, 2-speed, 3-small code, 4-compresion ratio to worth compressing.
Any guidance or article appreciated.
If operating system provides compresion and decompresion, It will be better.
Thanks in advance
mr.abzadeh
Posted
Updated 10-Apr-12 21:05pm
v5
Comments
Sergey Alexandrovich Kryukov 10-Apr-12 21:58pm    
"Better than... repeated byte" is understandable, if you mean "run-length encoding", but why "simpler then Zempel-Ziv"? What do you want, simplicity (but who knows what is that?) or speed?

How can you ask what OS provides is you did not share your platform with us? No, please add it in the tags.

--SA
mr.abzadeh 11-Apr-12 3:01am    
Thanks for your interest.
You are right, I should specify os, which is windows xp sp2 and above.
And I do not have good familiaity with compresion algorythms,
by counting repeated bytes I mean very simple algorythms, not run-length encoding, what I want is: 1-compress small arrays, 2-speed, 3-small code
enhzflep 11-Apr-12 1:53am    
Lzw can be implemented easily in a few hours or less, adding about 10kb to the exe for both compression/decompression. There's also zlib. This will add ~70kb to your exe.
mr.abzadeh 11-Apr-12 3:10am    
Thanks, lzw may be acceptable as it addes 10 kb, Does it compress small arrays? Can you offer me a reliable link to download lzw? Do you know another algorythm with smaller code size?
Sergey Alexandrovich Kryukov 12-Apr-12 12:53pm    
You really need to try. I would also try Huffman maybe...
--SA

The best alogorithm depends on the nature of your data. If they are, for instance, random values then you'll better don't even try to compress them.
If you need a guidance on a compression method then I suggest you the Salomon's book: "Data Compression: The Complete Reference"[^].

On the other hand, if you are searching for a ready-to-use library, then you have several options (they depend also on your favourite development platform, for instance, .NET Framework provides the System.IO.Compression Namespace[^]) and Google is you friend friend[^].
 
Share this answer
 
Comments
mr.abzadeh 11-Apr-12 3:57am    
Thanks for your answer.
I am searching in google know.
Since I code in native cpp, I can not use System.IO.Compression Namespace.
Is there any library for compresion in native cpp?
CPallini 11-Apr-12 4:31am    
There are many: Google for, Google for and Google for again!
Espen Harlinn 11-Apr-12 7:41am    
5'ed!
VJ Reddy 7-May-12 10:34am    
Good answer. 5!
CPallini 7-May-12 12:03pm    
Thank you.
I often use #ziplib [^] because of the wide range of supported formats - including encryption support.

[Update]
When I need something like this in C++ I look to the LZMA SDK[^]

Best regards
Espen Harlinn
 
Share this answer
 
v2
Comments
mr.abzadeh 11-Apr-12 10:33am    
Thanks for your reply.
These are good, but they compress large amount of data efficiently. I need an algorythm to compress small amount of data, from 100 bytes to 4k.
Espen Harlinn 11-Apr-12 10:39am    
You can always try something simple like:
http://en.wikipedia.org/wiki/Run-length_encoding
or something like http://en.wikipedia.org/wiki/LZ77_and_LZ78 with a predefined dictionary generated from sample data.

The first one is pretty easy to implement while the second one requires something more of an effort.
VJ Reddy 7-May-12 10:33am    
Good answer. 5!
Espen Harlinn 7-May-12 10:37am    
Thank you, VJ :-D
how about huffman encoding http://hbfs.wordpress.com/2011/05/17/huffman-codes/[^]. i have done it.but not sure about the speed of it. don't know whether this helps you.
 
Share this answer
 
Comments
mr.abzadeh 11-Apr-12 3:13am    
Thanks for your guidance.
I need lighter argorythm than huffman encoding, I want this: 1- compress small byte counts in memory, 2- good speed, 3- small code.
Does windows provide compression library?
jk chan 11-Apr-12 3:32am    
what is the minimum speed you want to achieve for your chunk(500bytes) ?

mr.abzadeh 11-Apr-12 3:53am    
I do not have an absolute minimum size.
I want to log Insert, update, deletes on a database
I want to collect the row's previous and next value, compress them and store in a log table. I can store small record, and compress larger ones, but since there are tens of milions of rows, speed and byte size as small as 100 bytes and even less is desirable,
Thanks

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900