PrimeGrid
Please visit donation page to help the project cover running costs for this month

Toggle Menu

Join PrimeGrid

Returning Participants

Community

Leader Boards

Results

Other

drummers-lowrise

Advanced search

Message boards : Primegen : Compressed

Author Message
suicid
Send message
Joined: 5 Feb 08
Posts: 9
ID: 18638
Credit: 12,656
RAC: 0

Message 8418 - Posted: 13 Mar 2008 | 17:58:53 UTC
Last modified: 13 Mar 2008 | 18:01:28 UTC

I compressed DVD 1,2,3,4 and 5 (download of DVD6 is not completed yet)
Disks 1-6 should be smaller than 8GB (only 2 DVDs)
--> Am I allowed to publish the files? If not I can publish only the 36 Bit version because that I got before I found primegrid... <--

The algorithm is not that complex... can be used without decompressing the data with a special tool/algorithm (like I had to do with the original and 80 - 100 GB are not funny if the files would fit on 2 DVDs and decompress took some hours, too)

I splitted (some files, each around 1 GB) and indexed (every 50.000th prime is in the index) the files... by this way it's possible to search for the x-th prime or to check if y is prime pretty fast...

MfG SuiCid
PS: I will have to check the algorithm again by decompressing the file and comparing the data with the original... (maybe I have to set some constants)
____________

Rytis
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 22 Jun 05
Posts: 2651
ID: 1
Credit: 37,151,315
RAC: 197,204
321 LLR Silver: Earned 100,000 credits (104,475)Cullen LLR Silver: Earned 100,000 credits (291,372)ESP LLR Bronze: Earned 10,000 credits (18,156)Generalized Cullen/Woodall LLR Bronze: Earned 10,000 credits (15,259)PPS LLR Silver: Earned 100,000 credits (114,027)PSP LLR Silver: Earned 100,000 credits (116,517)SoB LLR Silver: Earned 100,000 credits (282,655)SR5 LLR Bronze: Earned 10,000 credits (14,071)SGS LLR Silver: Earned 100,000 credits (100,082)TPS LLR (retired) Silver: Earned 100,000 credits (111,607)TRP LLR Sapphire: Earned 20,000,000 credits (31,441,028)Woodall LLR Silver: Earned 100,000 credits (101,463)321 Sieve (suspended) Silver: Earned 100,000 credits (201,501)Cullen/Woodall Sieve (suspended) Silver: Earned 100,000 credits (214,653)Generalized Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (14,200)PPS Sieve Silver: Earned 100,000 credits (491,193)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Silver: Earned 100,000 credits (200,232)TRP Sieve (suspended) Ruby: Earned 2,000,000 credits (2,453,872)AP 26/27 Silver: Earned 100,000 credits (473,058)WW Bronze: Earned 10,000 credits (60,000)GFN Silver: Earned 100,000 credits (197,387)PSA Bronze: Earned 10,000 credits (97,541)
Message 8422 - Posted: 13 Mar 2008 | 20:33:13 UTC - in response to Message 8418.

I compressed DVD 1,2,3,4 and 5 (download of DVD6 is not completed yet)
Disks 1-6 should be smaller than 8GB (only 2 DVDs)
--> Am I allowed to publish the files? If not I can publish only the 36 Bit version because that I got before I found primegrid... <--

The algorithm is not that complex... can be used without decompressing the data with a special tool/algorithm (like I had to do with the original and 80 - 100 GB are not funny if the files would fit on 2 DVDs and decompress took some hours, too)

I splitted (some files, each around 1 GB) and indexed (every 50.000th prime is in the index) the files... by this way it's possible to search for the x-th prime or to check if y is prime pretty fast...

MfG SuiCid
PS: I will have to check the algorithm again by decompressing the file and comparing the data with the original... (maybe I have to set some constants)

We are ending the lifecycle of primegen, so we'll publish the remaining files quite soon. DVD6 contains primes up to 210G, we stopped at 640G.

I'll be on holiday until Easter, so I won't be able to publish files until then; expect to see them right after I'm abck.
____________

suicid
Send message
Joined: 5 Feb 08
Posts: 9
ID: 18638
Credit: 12,656
RAC: 0

Message 8427 - Posted: 14 Mar 2008 | 13:46:17 UTC
Last modified: 14 Mar 2008 | 13:48:45 UTC

If you got a windows pc (guess you have) then I can give you the Delphi-program and the sourcecode... (I will make some changes...(*))
if not I have to rewrite compress in C(++)
should be better than sending you the files with max. 100kb/s via azureus (1d when I use the complete line) and you will need the program to compress the other disks...

640G --> ~23.542.625.051 primes ( 640G / ln(640G) )
index every 50.000th number: 7,184631348 MB index file
every 25.000th number: 15MB
every 10.000th number: 36MB

size of resdata will be needed on every disk... (to access the primes
each "Res*.dat"-file 4 byte

last number of each file --> 8 byte / file

primes:
min: 22 GB
max: 88 GB (disk 1-6 = 8 GB --> max should be smaller than 64 GB)
I guess maximum will be not more than 30 GB... (~7 DVDs)

(*) Autodeletion (with deactivate-option) for finished (original) files...
and some kind of resume... should be some GB to be compressed...
--> possibility to compress disk by disk or file by file or everything at once...
(you will see... ;) )

Rytis
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 22 Jun 05
Posts: 2651
ID: 1
Credit: 37,151,315
RAC: 197,204
321 LLR Silver: Earned 100,000 credits (104,475)Cullen LLR Silver: Earned 100,000 credits (291,372)ESP LLR Bronze: Earned 10,000 credits (18,156)Generalized Cullen/Woodall LLR Bronze: Earned 10,000 credits (15,259)PPS LLR Silver: Earned 100,000 credits (114,027)PSP LLR Silver: Earned 100,000 credits (116,517)SoB LLR Silver: Earned 100,000 credits (282,655)SR5 LLR Bronze: Earned 10,000 credits (14,071)SGS LLR Silver: Earned 100,000 credits (100,082)TPS LLR (retired) Silver: Earned 100,000 credits (111,607)TRP LLR Sapphire: Earned 20,000,000 credits (31,441,028)Woodall LLR Silver: Earned 100,000 credits (101,463)321 Sieve (suspended) Silver: Earned 100,000 credits (201,501)Cullen/Woodall Sieve (suspended) Silver: Earned 100,000 credits (214,653)Generalized Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (14,200)PPS Sieve Silver: Earned 100,000 credits (491,193)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Silver: Earned 100,000 credits (200,232)TRP Sieve (suspended) Ruby: Earned 2,000,000 credits (2,453,872)AP 26/27 Silver: Earned 100,000 credits (473,058)WW Bronze: Earned 10,000 credits (60,000)GFN Silver: Earned 100,000 credits (197,387)PSA Bronze: Earned 10,000 credits (97,541)
Message 8429 - Posted: 14 Mar 2008 | 13:51:56 UTC - in response to Message 8427.

If you got a windows pc (guess you have) then I can give you the Delphi-program and the sourcecode... (I will make some changes...(*))
if not I have to rewrite compress in C(++)
should be better than sending you the files with max. 100kb/s via azureus (1d when I use the complete line) and you will need the program to compress the other disks...

If it's Delphi 7 compatible files, please send me the sourcecode - I am developing business applications at work with Delphi 7, so it won't be a problem; otherwise send me the executable (make sure to compress it in a password protected archive if sending by mail.
____________

suicid
Send message
Joined: 5 Feb 08
Posts: 9
ID: 18638
Credit: 12,656
RAC: 0

Message 8517 - Posted: 17 Mar 2008 | 8:06:16 UTC

OK.... got the program... it should do the job...
I could not verify that it is perfect becuse my memory has 21 errors (found in one hour and everything in the 512 MB - 2 GB Area)
§$%&//$&//§/&%&&/%&)"§&"§ BEEEEEEEP $%&/"(§"/(§"§

--> I will not give you the program
--> you will have to compile the sourcecode yourself (it's written in Delphi 7)


Algorithm:
Is based on storing the difference of prime on position x and position x-1
Pos(0) = 1 (needed a start value even if it is not prime; in PGR (PrimeGrid Reader) it will be Pos(0) = 2)
Pos(1) = 3
Pos(2) = 5
.....
Pos(x) = prime on position x

Do for all primes (x = 1 to max):
current = Pos (x) - Pos (x - 1);
current = (current shr 1) - 1;
if (current < 254)
write (resultfile, byte(current));
else if (current < 253 + (2 shl 16)) then begin
write (resultfile, byte(254));
write (resultfile, word(current - 254));
end
else
write (resultfile, byte(255));
write (resultfile, longword (current - 254 - (2 shl 16)));
//triplebyte = last 3 byte of current
//longword is the easiest way to handle this...
//triplebyte --> max. difference ~ 33M
//longword --> max. difference ~ 8000M
end;


One buffer for input and one for output should be used...



My program:
Will send you the address via PM...
PGR (the reader) is not ready yet... got a little design...

correct order of the input needed
(3,5,7,11,13,..... is ok
5,3,13,7,11 is not ok)

Disks will look like this:
4 resultfiles (each about 1 GB)
1 index
some special files (filesize, ids,....)

e.g.:
Disk#1 content:
PG00.dat,PG01.dat,PG02.dat,PG03.dat,index.dat,....
Disk#2 content:
PG04.dat,PG05.dat,PG06.dat,PG07.dat,index.dat,....

if you want me to rename the files then just give me the name...




PGR:
PGR will use bisection on the index to find the highest value / id (=position)
that is stored in the loaded index... (index = 8 byte position + 8 byte value
(index will contain 72 MB or 144 MB --> there is an option to load only one half / third / ...
by loading every / every second / every third / ......)
Runtime: Only O(log (size(index / 16))) to find the position to start...

1. chose the right file by firstvalue/lastvalue or firstid/lastid #1
2. bisection on index - fast search --> position#2
3. from position#2 to the right primenumber (by value or by id)
....

(C) and idea by me... ;)

have to go... have to upload the data and send you the link...

suicid
Send message
Joined: 5 Feb 08
Posts: 9
ID: 18638
Credit: 12,656
RAC: 0

Message 8522 - Posted: 19 Mar 2008 | 20:38:42 UTC

Still busy... gave the address to the programs to Rytis...
the reader is not finished yet cause the memory of my normal pc is damaged...
--> compress should work - could not verify this 'cause some of the strings (from the original files) looked like this "23456$2424" or "%&/$""§SDF24"...

will try it on the laptop again... with 1-10 MB files instead of 1 GB and only 0.1 disk instead of 6 DVDs...
the procedures for using the files will be the same... I hope so... ;)

Diethelm
Send message
Joined: 8 May 07
Posts: 4
ID: 8041
Credit: 45
RAC: 0

Message 8535 - Posted: 22 Mar 2008 | 6:22:06 UTC

Hello suicid!

I watch this thread for a time. You are very engaged !

Perhaps this Info can help you:

Some times ago I found problems in the published files of PrimeGrid, too.

Look here in the forum: PrimGen
thread: Problems and Help : Published Results
Message: 5646

At that time I decided to spent no working time of my PC's to this
project ...
It may be that you find this interesting:

thread: End of the life cycle of primegen application...
Message 8457

Greetings from Borken, Germany

suicid
Send message
Joined: 5 Feb 08
Posts: 9
ID: 18638
Credit: 12,656
RAC: 0

Message 11013 - Posted: 7 Oct 2008 | 9:21:02 UTC

Stopped crunshing a long time ago (pc crashed, no time to fix it, then needed new hardware, .... no pc to run the program...only laptop...)

I wrote a program to get the numbers myself...
2^35 is already done
2^36 is about to be finished (less than a week)
2^37 will need some time (~ 1 month)
...

size
now: 2.2 GB ~ 2.2 GP
@2^37: >=5 GB, but less than 6 GB

[2^37 ~ 137.438.953.472 ~ DVD 1-4]
[x GP = x GigaPrimes = x * 1024^3 primes
too lazy to write large numbers and 1GB ~ 1GP ;) ]

will publish the results "when it's done"...
index-generator and viewer needed to use the files...
need to write 2 programs and the pc should finish the first 4.5 GB before I publish the files... and I want to run a test on another pc...

the programm will stop between 2^37 and 2^39
reason: time needed to generate the files
up to: time for new area | sum complete area | size of file
2^36: ~14 days | left: 0 ("done")|<2.7 GB
2^37: >30 days | left: >30 days | <6 GB
2^38: >60 days | left: >90 days | <11 GB
2^39: >120 days | left: >210 days | <20GB
2^40: >240 days | left: >450 days | <40 GB

Diethelm
Send message
Joined: 8 May 07
Posts: 4
ID: 8041
Credit: 45
RAC: 0

Message 11043 - Posted: 9 Oct 2008 | 11:01:44 UTC - in response to Message 11013.

Hello Folks !

My Prime Number Database is already online with
several functions and features, for example
accessable by a Google Gadget.

Take a look at http:/mathemacher.dph-software.de

At the time it's prime number range is more than
525 times larger than the published PrimeGen range
of 210.000.000.000 (i.e. > 110.630.000.000.000).


Greetings from Borken

Germany
____________

suicid
Send message
Joined: 5 Feb 08
Posts: 9
ID: 18638
Credit: 12,656
RAC: 0

Message 56204 - Posted: 8 Jul 2012 | 14:19:58 UTC

new pc(s)... more power...and not that much time to use them...
40 bit should be (almost) done or even done... 100% tested, not only 99,999999...% by Miller-Rabin...

but I guess I will stop here because it's more than I need...
less than Diethelms DB, more than primegrid DB... :D

Message boards : Primegen : Compressed

[Return to PrimeGrid main page]
DNS Powered by DNSEXIT.COM
Copyright © 2005 - 2021 Rytis Slatkevičius (contact) and PrimeGrid community. Server load 2.43, 2.84, 2.65
Generated 3 Mar 2021 | 14:47:15 UTC