Selecting the hardware backend
You must specify a device via the --device option. Multiple devices can be used together. For that, multiple --device options must be given.
Shutdown/Resume
If you run the command line that was given to you by the generator cgi, you will start writing to the file "data" in the current directory (option consume:file:prefix). If you need to shutdown, you can either kill the process or ctrl+c it, or you can ask it to complete all currently worked on chains first (takes a minute to shutdown then). For graceful shutdown, execute
$ a51table signal --shutdown
in the directory where you started the program. If you want to resume generating, start the original command and it will append data to the already generated table (see the consume:append option). So it makes sense to paste the command line into a shell script. you can also write to more than one file, by changing to consume=file:prefix option and later read from multiple files (for example data*) when sorting the chains.
Speed and User Interface Responsiveness
If you increase the device:cuda:operations option you can squeeze out some percent of performance but the user interface suffers. Too many operations are not advisable, because chains that reached their end are only sorted out on the CPU. device:cuda:operations should be between 256 and 8192. intermediate:filter:runlength *must* be larger than device:cuda:operations and *should really* be the same value to not waste resources.
Checking values against the reference implementation
$ make test
This checks against a small table shipped with the program in data/testdata. Alternatively,
There is a minimal implementation of the chain generation process in reference/a51.cpp and you can compile it with
$ cd obj; make refa51
To check values against it, either use the work_consumer/print with --consume print or od(1) some data out of the generated tables with for example
$ od -t x8 -w8 -N 8 data.start.tbl
The command to run is then:
refa51 0x64_bit_start_value 3000000 15 32 your_advance_parameter
Configuration files
the program supports simple configuration file. the syntax is:
name = value
for options taking a value and simply
name
for boolean options. name is the same as the corresponding option name without the "--". There is a configuration file built into the program that computes tables suitable to this project. To list all options that are effectivly used, run the program with --options, as in
prog --options generate --chains 1024
To load a particular configuration file, use
$ prog --config <filename> --other --options with_values command --more --options
options on the command line override options in the configuration file, those in the configuration file override the ones in the executable.
There is a new option "--advance" which has the same meaning as the advance option of the round function generator, so you can run:
$ prog --advance 23456 generate
to create tables for this project with all other values at defaults.
Running
$ prog --options generate
outputs
--algorithm A51 --condition rounds:rounds=32 --consume print --device cuda --implementation sharedmem --logger normal --options --roundfunc xor:condition=distinguished_point::bits=15:generator=lfsr::tablesize=32 --work random:prefix=11,0 generate --chainlength 3000000 --chains 400000000 --intermediate filter:runlength=512
as default values. So you have to add --advance XXX and --consume file:prefix=my_file and possible --operations XXX.
There is another new option "--operations" which sets the operations option of the device option as well as the runlength option of the intermediate option simultaneously.
This is how a configuration file with default values would look like:
condition = rounds:rounds=32 roundfunc = xor:condition=distinguished_point::bits=15:generator=lfsr::tablesize=32 logger = verbose device = cuda work = random:prefix=11,0 consume = file:prefix=data:append algorithm = A51 implementation = sharedmem operations = 512 advance = 0 !generate chains = 400000000 chainlength = 3000000 intermediate = filter
Signaling
To influence a running process, execute (in the directory with the "signaling" fifo)
$ prog signal --shutdown
to shut it down. Execute
$ prog signal --operations N
to change the value of the operations option of the device option, i.e. the number of A5/1 rounds computed per cuda kernel call. This also changes the runlength parameter of the filter work intermediate, to the same value.
the parameter N can either be the number of kernel operations, +N or -N to add/substract from the current value, or +N%, -N% to add/substract a percentage from the current value. (the percentage is calculated relative to the current value).
$ prog signal --pause
Halts computation (but keeps all intermediate chains in ram), until --pause is signaled again.
File sets
A file set is simply a collection of table files that have the same parameters. It is represented by a file that lists all files that belong to a file set. Another feature is the automatic splitting of tables that are being generated into more than one file. For that a desired file size and a template to generate the file names can be defined in the fileset configuration file.
Table Generation status
During table generation a http request is sent regularly to a central server to report the amount of work done so far and the current rate of operations. The protocol is simple enough to be inspected by humans to verify what is being transmitted. This is the advance parameter the chain rate and the amount of chains generated. For these status updates to be able to report the current amount of chains already generated, the work consumer must either be a single file that contains all the chains or it must be a fileset referencing all the chains, so the program knows which data files to account for. To enable network updates, set the --network option.
--network nickname=me_myself_and_i:password=12345:host=reflextor.com:port=80:htproxy=localhost^8118:interval=20m
the htproxy is optional, you can use it to send the network updates via privoxy through tor. torify will not work (seems to have a problem with async connect). The ^ is a placeholder for the usual : in host:port notation. The server answers with "Good work!" if the update was successful. In the native windows shell (not cygwin) you have to write host^^port. To use socks4/socks4a natively, instead of htproxy=host^port, write socks4=host^port or socks4a=host^port. socks4 is not recommended, since the DNS lookups will not be onion-routed.
Report finished work
$ a51table --advance XXX --work file:prefix=data* --consume null --network simple:nickname=foo:password=bar:atend:reset copy
if there is a complaint about a missing .table file, simply touch it, the content is not used right now.
Running with multiple GPUs
If you have more than one graphics adapter or a multi GPU card like the GTX295, the table generator can use all of them automatically and would generate a single table with all of them. If you want to generate multiple tables you would have to start a process for each table and GPU manually. To enable the automatic multicard support, supply a --device option for each GPU.
$ a51table --device cuda:devno=0 --device cuda:devno=1 generate
To list the available devices:
$ a51table --device cuda:list generate
Verifying a random subset of a table
$ a51table --work file:prefix=data23:select=random,8192:startonly --consume compare generate
8192 is the number of chains to randomly select to recompute and verify. To look at actual chain values, use:
$ a51table --work like_above --consume print:compare[:all][:results=n] generate
or
$ a51table --work like_above --consume compare:match=print::all:nomatch=print::results=235 generate
Sorting a table
There is a shell script available that automates the sequence of commands below at http://reflextor.com/sort_a51.sh
For now you also need another binary to sort (64bit linux only) http://reflextor.com/sorta51table
When a table is complete, it has to be sorted. The command to sort a table is:
$ a51table --advance $advance --work file:prefix=data --icondition accept --device shortcircuit --consume $consume_opt copy --intermediate sort:parts=$parts:final:ram=$ram
where $consume_opt is
file:prefix=sorted_data:startbits=53:truncstart=msb:endbits=49:truncend=lsb:indexbits=25
startbits=53 assumes that you used the default --work option while generating which set the topmost 11 bits to zero endbits=49 truncates 15bit of distinguished point which are always zero
if you created non-standard tables then you must give the condition and roundfunc options to the sorter, e.g. supply --condition rounds:rounds=8 as a parameter.
$parts and $ram depend on the number of chains you want to sort. for the sorter a chain needs 32byte ram. if you have 1gb available, then set ram=1024 and parts = number_of_chains/32million then increase the number of parts a bit to be on the safe side. or, set parts to the number of million chains and ram to something above 32 to compensate for rounding errors. if you do not assign enough ram or parts to the sorter then it will invoke stxxl and take longer.
If you have more then one file created during table generation, you can specify a glob expression which must not be evaluated by the shell.
$ a51table --work file:prefix=data\*
to verify that everything went as planned run:
$ a51table --work file:prefix=data_sorted:startbits=53:truncstart=msb:endbits=49:truncend=lsb:indexbits=25:skip=$N:limit=10 --device shortcircuit copy
and run a few chains through the reference implementation. if you use the optional skip=$N set $N to the number of chains to seek to before printing.
If you sort a table that was created with the old buggy version released before October 25th, then you should still share your table. Follow the steps above and manually change the string "lfsr2" to "lfsr" in the sorted.table file.
The layout of the torrent should be as such:
Name of the directory: "a51_rt_$(advance-paramter)" Files inside the directory: sorted.start.tbl sorted.end.tbl sorted.index sorted.table
The torrent trackers should include one public and a hidden service tracker
- http://tracker.openbittorrent.com/announce
- udp://tracker.openbittorrent.com:80/announce
- http://z6gw6skubmo2pj43.onion:8080/announce
- http://z6gw6skubmo2pj43.tor2web.com:8080/announce
the proper mktorrent command line would thus be (with $1 being your advance parameter):
$ mktorrent -l 21 -n a51_rt_$1 -a http://z6gw6skubmo2pj43.onion:8080/announce,http://z6gw6skubmo2pj43.tor2web.com:8080/announce,http://tracker.openbittorrent.com/announce,udp://tracker.openbittorrent.com:80/announce a51_rt_$1
