[zfec-dev] About using zfec

[zooko]

Hello Tanzima Zerin Islam:

> 1. how to use the command line utility zfec?

This is described in the README.txt file.  If the instructions in  
there are inadequate please let me know why and I'll try to improve  
them.


> 3. I figured, I probably have to write wrappers around the C file.  
> But my question is:
> --- I need to have a module where I will provide with the file name  
> and k, M, and get back an array of pointer to erasure coded data.
> Is it possible to do with the C API of the software? The reason why  
> I am asking this question, is that there is another module named
> filefec.py that is supposed to read the input file, and divide it  
> into K chunks? So If I have to erasure code a file, then first of all
> I have to run that script to divide it into blocks and then feed  
> the returned blocks (that are called primary blocks?) to fec_encode 
> () to
> erasure code those blocks?

This too should be answered by the README.txt.  Hm, I think I can see  
how to make it clearer...

Okay, I've added the following paragraph to the README.txt:

"""
Note that this fec_encode() is a "low-level" API in that it requires
the input data to be provided in a set of memory buffers of exactly
the right sizes.  If you are starting instead with a single buffer
containing all of the data then please see easyfec.py's "class
Encoder" as an example of how to split a single large buffer into the
appropriate set of input buffers for fec_encode().  If you are
starting with a file on disk, then please see filefec.py's
encode_file_stringy_easyfec() for an example of how to read the data
from a file and pass it to "class Encoder".  The Python interface
provides these higher-level operations, as does the Haskell interface.
If you implement functions to do these higher-level tasks in other
languages than Python or Haskell, then please send a patch to
zfec-dev at allmydata.org so that your API can be included in future
releases of zfec.
"""

So, basically, I'm asking you to please write the Java equivalent of  
Python's easyfec.py (which takes an arbitrary-length string and  
splits it up into packets and passes the packets to fec_encode()),  
and filefec.py (which takes an input file, iteratively reads some  
bytes out of it and passes those bytes to easyfec), or Haskell's  
"enFEC" function:

http://allmydata.org/trac/zfec/browser/zfec/haskell/Codec/FEC.hs

And, once you've done that and written unit tests and documentation  
for it, to please submit a patch to this list.  :-)

Also, if you prefer, you could write the C equivalent of easyfec and  
filefec and then invoke those "high-level" C functions from Java.


> 3. My software written in JAVA and I have to use Jni programming to  
> call the fec_encode() method of C. My question is, is it absolutely  
> necessary to
> install the software for encoding?

Certainly not.  There was a comment in the README.txt saying that you  
"must install the software", but that was saying that this is  
necessary in order to run the Haskell unit tests, not in order to use  
the software at all.  I've amended that part of the README.txt to be  
clearer.


> Tan
> Ps: Sorry if I am sending this email multiple times.

That was my fault -- the zfec-dev list's settings were set wrong so  
that it prevented you from posting, and then when I fixed it I  
accidentally sent three copies of your post through.

Regards,

Zooko
---
http://allmydata.org -- Tahoe, the Least-Authority Filesystem
http://allmydata.com -- back up all your files for $5/month