PrimeGrid
Please visit donation page to help the project cover running costs for this month

Toggle Menu

Join PrimeGrid

Returning Participants

Community

Leader Boards

Results

Other

drummers-lowrise

Advanced search

Message boards : Number crunching : Too much work when joining

Author Message
Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 145141 - Posted: 10 Nov 2020 | 16:49:48 UTC

When I attached my computers to Primegrid, with them set to a 0+0.13 day buffer (0+3 hours), why did I receive an estimated 2 weeks of work? It actually looks like it's 1 week of work, so the estimate is out by a bit. But what caused the server to think 1 week of work, estimated at 2 weeks, would fit in my 3 hour buffer?

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149785 - Posted: 1 Apr 2021 | 10:43:30 UTC - in response to Message 145141.
Last modified: 1 Apr 2021 | 10:43:57 UTC

And still it continues after being joined for months. Asks for 3 hours of work, gets a week. What a waste of server resources with half your tasks being aborted as they don't get run on time. Fix. The. Scheduler. You're the only project that does this.

Profile j.sheridanProject donor
Volunteer tester
Send message
Joined: 21 Mar 11
Posts: 816
ID: 91622
Credit: 2,331,853,674
RAC: 2,280,044
Discovered 2 mega primesFound 4 primes in the 2021 Tour de PrimesFound 2 mega primes in the 2021 Tour de Primes321 LLR Jade: Earned 10,000,000 credits (10,011,712)Cullen LLR Jade: Earned 10,000,000 credits (10,025,728)ESP LLR Jade: Earned 10,000,000 credits (10,176,490)Generalized Cullen/Woodall LLR Jade: Earned 10,000,000 credits (10,020,580)PPS LLR Sapphire: Earned 20,000,000 credits (29,059,451)PSP LLR Jade: Earned 10,000,000 credits (10,055,994)SoB LLR Sapphire: Earned 20,000,000 credits (39,949,623)SR5 LLR Jade: Earned 10,000,000 credits (10,885,368)SGS LLR Jade: Earned 10,000,000 credits (10,007,143)TRP LLR Jade: Earned 10,000,000 credits (10,009,531)Woodall LLR Jade: Earned 10,000,000 credits (10,039,527)321 Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,019,388)Cullen/Woodall Sieve (suspended) Double Silver: Earned 200,000,000 credits (265,102,350)PPS Sieve Double Gold: Earned 500,000,000 credits (547,992,812)TRP Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,801,812)AP 26/27 Emerald: Earned 50,000,000 credits (93,546,934)GFN Double Silver: Earned 200,000,000 credits (436,635,654)WW Double Gold: Earned 500,000,000 credits (802,548,000)
Message 149791 - Posted: 1 Apr 2021 | 12:33:13 UTC - in response to Message 149785.
Last modified: 1 Apr 2021 | 12:33:57 UTC

Your tasks would run a hell of a lot faster if you switch your options to 50% processors (you've got hyperthreading on) and on the 3900XT you'll need to check but it will be anything up to 10x faster using the correct number of cores per task based on the FFT size, and will likely be a maximum of 6.
Not sure about the xeon but you can do the same prime95 benchmark on that to find the optimum number of cores per task.

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149795 - Posted: 1 Apr 2021 | 18:06:28 UTC - in response to Message 149791.

Your tasks would run a hell of a lot faster if you switch your options to 50% processors (you've got hyperthreading on)

I thought hyperthreading, although not doubling the speed, will increase the overall speed of the whole CPU. In my experience you get about 1.5x the speed. So although turning it off may make each single core task run faster, the overall throughput of the machine drops. If it was as useless as you suggest, why did AMD and Intel invent it?

and on the 3900XT you'll need to check but it will be anything up to 10x faster using the correct number of cores per task based on the FFT size, and will likely be a maximum of 6.
Not sure about the xeon but you can do the same prime95 benchmark on that to find the optimum number of cores per task.

Please give me more info on this. Are you saying Primegrid is issuing tasks with the wrong number of cores set? Why would they do that? And what exactly do I run to find the optimal number of cores to use? And will it be similar for all the subprojects?

Profile Rick Reynolds
Avatar
Send message
Joined: 13 Aug 10
Posts: 8159
ID: 65483
Credit: 6,711,828,284
RAC: 40,792
Discovered 3 mega primesFound 2 primes in the 2018 Tour de PrimesFound 2 primes in the 2019 Tour de PrimesFound 12 primes in the 2020 Tour de PrimesFound 2 primes in the 2021 Tour de PrimesFound 1 mega prime in the 2021 Tour de Primes321 LLR Jade: Earned 10,000,000 credits (10,244,909)Cullen LLR Jade: Earned 10,000,000 credits (10,194,451)ESP LLR Turquoise: Earned 5,000,000 credits (9,538,687)Generalized Cullen/Woodall LLR Jade: Earned 10,000,000 credits (10,216,354)PPS LLR Emerald: Earned 50,000,000 credits (73,178,392)PSP LLR Turquoise: Earned 5,000,000 credits (5,151,448)SoB LLR Jade: Earned 10,000,000 credits (10,045,557)SR5 LLR Jade: Earned 10,000,000 credits (14,871,275)SGS LLR Turquoise: Earned 5,000,000 credits (7,860,522)TRP LLR Jade: Earned 10,000,000 credits (11,130,742)Woodall LLR Turquoise: Earned 5,000,000 credits (5,419,909)321 Sieve (suspended) Double Bronze: Earned 100,000,000 credits (101,538,236)Cullen/Woodall Sieve (suspended) Double Bronze: Earned 100,000,000 credits (127,922,665)Generalized Cullen/Woodall Sieve (suspended) Sapphire: Earned 20,000,000 credits (30,984,158)PPS Sieve Double Ruby: Earned 2,000,000,000 credits (2,840,173,136)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Turquoise: Earned 5,000,000 credits (8,134,964)TRP Sieve (suspended) Turquoise: Earned 5,000,000 credits (7,013,696)AP 26/27 Double Silver: Earned 200,000,000 credits (276,723,135)GFN Double Silver: Earned 200,000,000 credits (349,277,565)WW Double Amethyst: Earned 1,000,000,000 credits (1,431,508,000)PSA Double Amethyst: Earned 1,000,000,000 credits (1,370,700,486)
Message 149796 - Posted: 1 Apr 2021 | 18:58:24 UTC

Is it possible you are getting hyper-threading and multi-threading mixed up? I know I did alot on the front end when multi-threading became available.

PG looks at your pc in terms of cores so if you have ht turned on for a 2 core system it thinks you have 4 cores when you really only have 2 physical cores. Most of the projects will run faster on a physical core vs a ht core.

Multi-threading will allow you to configure the # of physical cores you want to run per task. So if you have an 8 core (physical cores) system and say you have your settings for mt so that you will use 4 cores per task it will effectively take 1/4 of the time to run one task (mileage varies alot) but you only run 2 tasks at once instead of say 8 (if you are allowing all 8 cores to be used).

Settings to use MT are in the preferences and you can set the number of tasks to get in total as well as the number of cores to use per task. Might want to keep in mind that the setting for total number of tasks will affect all tasks including GPU so make sure you have the number set to get enough for both cpu and gpu if you need to. MT setting only applies to the cpu cores.

Since you've been a member for a while you might already know and are asking a different question.

Cheers

Profile j.sheridanProject donor
Volunteer tester
Send message
Joined: 21 Mar 11
Posts: 816
ID: 91622
Credit: 2,331,853,674
RAC: 2,280,044
Discovered 2 mega primesFound 4 primes in the 2021 Tour de PrimesFound 2 mega primes in the 2021 Tour de Primes321 LLR Jade: Earned 10,000,000 credits (10,011,712)Cullen LLR Jade: Earned 10,000,000 credits (10,025,728)ESP LLR Jade: Earned 10,000,000 credits (10,176,490)Generalized Cullen/Woodall LLR Jade: Earned 10,000,000 credits (10,020,580)PPS LLR Sapphire: Earned 20,000,000 credits (29,059,451)PSP LLR Jade: Earned 10,000,000 credits (10,055,994)SoB LLR Sapphire: Earned 20,000,000 credits (39,949,623)SR5 LLR Jade: Earned 10,000,000 credits (10,885,368)SGS LLR Jade: Earned 10,000,000 credits (10,007,143)TRP LLR Jade: Earned 10,000,000 credits (10,009,531)Woodall LLR Jade: Earned 10,000,000 credits (10,039,527)321 Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,019,388)Cullen/Woodall Sieve (suspended) Double Silver: Earned 200,000,000 credits (265,102,350)PPS Sieve Double Gold: Earned 500,000,000 credits (547,992,812)TRP Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,801,812)AP 26/27 Emerald: Earned 50,000,000 credits (93,546,934)GFN Double Silver: Earned 200,000,000 credits (436,635,654)WW Double Gold: Earned 500,000,000 credits (802,548,000)
Message 149800 - Posted: 1 Apr 2021 | 20:54:37 UTC - in response to Message 149795.
Last modified: 1 Apr 2021 | 21:02:25 UTC

Your tasks would run a hell of a lot faster if you switch your options to 50% processors (you've got hyperthreading on)

I thought hyperthreading, although not doubling the speed, will increase the overall speed of the whole CPU. In my experience you get about 1.5x the speed. So although turning it off may make each single core task run faster, the overall throughput of the machine drops. If it was as useless as you suggest, why did AMD and Intel invent it?

LLR is highly optimised so doesn't benefit from HT. Other things are less optimised and do benefit from HT which is why it exists.


and on the 3900XT you'll need to check but it will be anything up to 10x faster using the correct number of cores per task based on the FFT size, and will likely be a maximum of 6.
Not sure about the xeon but you can do the same prime95 benchmark on that to find the optimum number of cores per task.

Please give me more info on this. Are you saying Primegrid is issuing tasks with the wrong number of cores set? Why would they do that? And what exactly do I run to find the optimal number of cores to use? And will it be similar for all the subprojects?

every cpu is different so they leave it up to you to choose the optimum settings for the cpu you're using.
Look for the thread on here on multithreading and using Prime95 benchmarks to determine the optimal number of cores for each FFT size.


Here are some timings I did on the 3950x for (older) SoB sized tasks:

Timings for 2880K all-complex FFT length (16 cores, 1 worker): Throughput: 880.28 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 2 workers): Throughput: 1116.63 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 4 workers): Throughput: 544.44 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 8 workers): Throughput: 406.65 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 16 workers): Throughput: 388.19 iter/sec.

Running 16 threaded tasks is 20% slower than running 2x8 threaded tasks, and for those crazy enough to run 16 at a time that's nearly 70% slower.

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149807 - Posted: 2 Apr 2021 | 11:19:54 UTC - in response to Message 149796.

Is it possible you are getting hyper-threading and multi-threading mixed up? I know I did alot on the front end when multi-threading became available.

PG looks at your pc in terms of cores so if you have ht turned on for a 2 core system it thinks you have 4 cores when you really only have 2 physical cores. Most of the projects will run faster on a physical core vs a ht core.

Multi-threading will allow you to configure the # of physical cores you want to run per task. So if you have an 8 core (physical cores) system and say you have your settings for mt so that you will use 4 cores per task it will effectively take 1/4 of the time to run one task (mileage varies alot) but you only run 2 tasks at once instead of say 8 (if you are allowing all 8 cores to be used).

Settings to use MT are in the preferences and you can set the number of tasks to get in total as well as the number of cores to use per task. Might want to keep in mind that the setting for total number of tasks will affect all tasks including GPU so make sure you have the number set to get enough for both cpu and gpu if you need to. MT setting only applies to the cpu cores.

Since you've been a member for a while you might already know and are asking a different question.

Cheers

I use CPU and GPU and pretty much all the subprojects (I only turn some off for a dodgy GPU that crashes with anything but arithmetic progressions). So the total number of tasks setting gets rather confusing.

I think what you were trying to say above is that Primegrid could be giving me more than I can do in the time I request because I've got 24 HT threads and not 24 cores. But I thought the server goes by how fast I've actually done them in the past, not just a number of cores. And it's been on the machine for a few months at least, so has had plenty of time to get an average. Also, more importantly, I'm getting a huge amount more than I ask. When the Ryzen requests 3 hours of CPU work (that's my buffer setting), I get at least 1 task from every single subproject at once, some of them several tasks, totalling about a week's work. A week is a lot more than 3 hours.

It could be 3 days, which would be out by a factor of 24, meaning the server or client is getting mixed up between time per CPU and time per core. I've had this discussion over in the main Boinc forum before, and I think the conclusion was it's a Boinc bug and related to Primegrid using outdated Boinc server software. Why haven't they upgraded?

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149808 - Posted: 2 Apr 2021 | 11:28:49 UTC - in response to Message 149800.

Your tasks would run a hell of a lot faster if you switch your options to 50% processors (you've got hyperthreading on)

I thought hyperthreading, although not doubling the speed, will increase the overall speed of the whole CPU. In my experience you get about 1.5x the speed. So although turning it off may make each single core task run faster, the overall throughput of the machine drops. If it was as useless as you suggest, why did AMD and Intel invent it?

LLR is highly optimised so doesn't benefit from HT. Other things are less optimised and do benefit from HT which is why it exists.


and on the 3900XT you'll need to check but it will be anything up to 10x faster using the correct number of cores per task based on the FFT size, and will likely be a maximum of 6.
Not sure about the xeon but you can do the same prime95 benchmark on that to find the optimum number of cores per task.

Please give me more info on this. Are you saying Primegrid is issuing tasks with the wrong number of cores set? Why would they do that? And what exactly do I run to find the optimal number of cores to use? And will it be similar for all the subprojects?

every cpu is different so they leave it up to you to choose the optimum settings for the cpu you're using.
Look for the thread on here on multithreading and using Prime95 benchmarks to determine the optimal number of cores for each FFT size.


Here are some timings I did on the 3950x for (older) SoB sized tasks:

Timings for 2880K all-complex FFT length (16 cores, 1 worker): Throughput: 880.28 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 2 workers): Throughput: 1116.63 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 4 workers): Throughput: 544.44 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 8 workers): Throughput: 406.65 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 16 workers): Throughput: 388.19 iter/sec.

Running 16 threaded tasks is 20% slower than running 2x8 threaded tasks, and for those crazy enough to run 16 at a time that's nearly 70% slower.

I'm in over my head here. How do I interpret your table above? Are you saying that one task from Primegrid allocated to 16 cores actually takes over twice as long to complete than the same task given only 2 cores? I must be misunderstanding that, because I can't believe allowing a program access to more cores could possibly slow it down.

I still have no idea whatsoever how to proceed. As they say on reddit, explain it like I'm five. I have a 24 thread (12 core) Ryzen 9 3900XT, and two computers each with dual CPU 6 core/12 thread Xeon X5650. What do I do with Prime95 to get the most efficient number of threads to give the tasks, then where do I put that in the settings?

Profile j.sheridanProject donor
Volunteer tester
Send message
Joined: 21 Mar 11
Posts: 816
ID: 91622
Credit: 2,331,853,674
RAC: 2,280,044
Discovered 2 mega primesFound 4 primes in the 2021 Tour de PrimesFound 2 mega primes in the 2021 Tour de Primes321 LLR Jade: Earned 10,000,000 credits (10,011,712)Cullen LLR Jade: Earned 10,000,000 credits (10,025,728)ESP LLR Jade: Earned 10,000,000 credits (10,176,490)Generalized Cullen/Woodall LLR Jade: Earned 10,000,000 credits (10,020,580)PPS LLR Sapphire: Earned 20,000,000 credits (29,059,451)PSP LLR Jade: Earned 10,000,000 credits (10,055,994)SoB LLR Sapphire: Earned 20,000,000 credits (39,949,623)SR5 LLR Jade: Earned 10,000,000 credits (10,885,368)SGS LLR Jade: Earned 10,000,000 credits (10,007,143)TRP LLR Jade: Earned 10,000,000 credits (10,009,531)Woodall LLR Jade: Earned 10,000,000 credits (10,039,527)321 Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,019,388)Cullen/Woodall Sieve (suspended) Double Silver: Earned 200,000,000 credits (265,102,350)PPS Sieve Double Gold: Earned 500,000,000 credits (547,992,812)TRP Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,801,812)AP 26/27 Emerald: Earned 50,000,000 credits (93,546,934)GFN Double Silver: Earned 200,000,000 credits (436,635,654)WW Double Gold: Earned 500,000,000 credits (802,548,000)
Message 149809 - Posted: 2 Apr 2021 | 12:38:11 UTC - in response to Message 149808.

What the table is saying is if 1 task is assigned to 16 cores and takes 1 minute to run, 2 tasks with 8 cores each will take ~90s or 45s per task, 4 tasks with 4 cores each will take 400s or 100s/task and so on.

To do the benchmarking: Download prime95. Run prime95. Find the "benchmark" tab. Stick in the FFT size of the sub project you want to run. eg for this task it's 2304K and you'll get this sort of output, obviously with 2304K rather than 2880K:

Timings for 2880K all-complex FFT length (16 cores, 1 worker): Throughput: 880.28 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 2 workers): Throughput: 1116.63 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 4 workers): Throughput: 544.44 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 8 workers): Throughput: 406.65 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 16 workers): Throughput: 388.19 iter/sec.

The one with the highest iter/sec is your optimal option. I've set primegrid to use 50% cpu, so only 'real' cores, and when running SoB I tell it to use a maximum of 8 threads.
Yes, it can be slower when using more cores because the architecture of the CPU means it's then using RAM in preference to cache which is much slower. With your 3900XT you'll likely be setting a maximum of 6 threads for big tasks and 3 threads for the smaller (< 2M ones).
Once that's done for the AMD you'll find another 10-20% performance by ensuring the tasks stay within the ccx/ccd they're assigned to by using a tool such as process lasso - if windows decides to swap cores across a ccx/ccd boundary it has to do so in ram.

Profile dannyridel
Volunteer tester
Avatar
Send message
Joined: 3 Feb 19
Posts: 919
ID: 1097922
Credit: 20,030,100
RAC: 24,214
Discovered 1 mega prime321 LLR Gold: Earned 500,000 credits (506,806)Cullen LLR Silver: Earned 100,000 credits (112,212)ESP LLR Silver: Earned 100,000 credits (255,803)Generalized Cullen/Woodall LLR Gold: Earned 500,000 credits (509,419)PPS LLR Ruby: Earned 2,000,000 credits (2,001,172)PSP LLR Gold: Earned 500,000 credits (616,940)SoB LLR Gold: Earned 500,000 credits (533,625)SR5 LLR Silver: Earned 100,000 credits (108,281)SGS LLR Gold: Earned 500,000 credits (501,474)TRP LLR Silver: Earned 100,000 credits (146,999)Woodall LLR Silver: Earned 100,000 credits (117,864)321 Sieve (suspended) Gold: Earned 500,000 credits (506,814)Generalized Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (30,033)PPS Sieve Ruby: Earned 2,000,000 credits (2,808,043)AP 26/27 Amethyst: Earned 1,000,000 credits (1,273,545)GFN Ruby: Earned 2,000,000 credits (3,480,035)WW Turquoise: Earned 5,000,000 credits (6,148,000)PSA Silver: Earned 100,000 credits (373,034)
Message 149810 - Posted: 2 Apr 2021 | 13:24:16 UTC
Last modified: 2 Apr 2021 | 13:24:27 UTC

This post gives a pretty good outline of how to use Prime95.
____________
SHSIDElectronicsGroup@outlook.com

waiting for a TdP prime...
Proth "SoB": 44243*2^440969+1


Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149814 - Posted: 2 Apr 2021 | 17:58:19 UTC - in response to Message 149809.
Last modified: 2 Apr 2021 | 17:59:23 UTC

What the table is saying is if 1 task is assigned to 16 cores and takes 1 minute to run, 2 tasks with 8 cores each will take ~90s or 45s per task, 4 tasks with 4 cores each will take 400s or 100s/task and so on.

To do the benchmarking: Download prime95. Run prime95. Find the "benchmark" tab. Stick in the FFT size of the sub project you want to run. eg for this task it's 2304K and you'll get this sort of output, obviously with 2304K rather than 2880K:

Timings for 2880K all-complex FFT length (16 cores, 1 worker): Throughput: 880.28 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 2 workers): Throughput: 1116.63 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 4 workers): Throughput: 544.44 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 8 workers): Throughput: 406.65 iter/sec.
Timings for 2880K all-complex FFT length (16 cores, 16 workers): Throughput: 388.19 iter/sec.

The one with the highest iter/sec is your optimal option. I've set primegrid to use 50% cpu, so only 'real' cores, and when running SoB I tell it to use a maximum of 8 threads.
Yes, it can be slower when using more cores because the architecture of the CPU means it's then using RAM in preference to cache which is much slower. With your 3900XT you'll likely be setting a maximum of 6 threads for big tasks and 3 threads for the smaller (< 2M ones).
Once that's done for the AMD you'll find another 10-20% performance by ensuring the tasks stay within the ccx/ccd they're assigned to by using a tool such as process lasso - if windows decides to swap cores across a ccx/ccd boundary it has to do so in ram.

Surely if the CPU usage in task manager is near 100%, then it's using the CPU fully? That's what I do with GPU work, if the GPU isn't near 100% usage, I run more on it at a time.

If I have to work it out seperately for every single subproject I don't think I'll bother. Is there a rough guideline for a CPU like mine?

Profile j.sheridanProject donor
Volunteer tester
Send message
Joined: 21 Mar 11
Posts: 816
ID: 91622
Credit: 2,331,853,674
RAC: 2,280,044
Discovered 2 mega primesFound 4 primes in the 2021 Tour de PrimesFound 2 mega primes in the 2021 Tour de Primes321 LLR Jade: Earned 10,000,000 credits (10,011,712)Cullen LLR Jade: Earned 10,000,000 credits (10,025,728)ESP LLR Jade: Earned 10,000,000 credits (10,176,490)Generalized Cullen/Woodall LLR Jade: Earned 10,000,000 credits (10,020,580)PPS LLR Sapphire: Earned 20,000,000 credits (29,059,451)PSP LLR Jade: Earned 10,000,000 credits (10,055,994)SoB LLR Sapphire: Earned 20,000,000 credits (39,949,623)SR5 LLR Jade: Earned 10,000,000 credits (10,885,368)SGS LLR Jade: Earned 10,000,000 credits (10,007,143)TRP LLR Jade: Earned 10,000,000 credits (10,009,531)Woodall LLR Jade: Earned 10,000,000 credits (10,039,527)321 Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,019,388)Cullen/Woodall Sieve (suspended) Double Silver: Earned 200,000,000 credits (265,102,350)PPS Sieve Double Gold: Earned 500,000,000 credits (547,992,812)TRP Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,801,812)AP 26/27 Emerald: Earned 50,000,000 credits (93,546,934)GFN Double Silver: Earned 200,000,000 credits (436,635,654)WW Double Gold: Earned 500,000,000 credits (802,548,000)
Message 149816 - Posted: 2 Apr 2021 | 18:52:07 UTC - in response to Message 149814.

You don't have to do anything if you don't want to, it still runs just slowly. I just figure given how much running 24x7 costs in electricity you'd want to get maximum bang for your buck.

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149823 - Posted: 3 Apr 2021 | 10:52:08 UTC - in response to Message 149816.
Last modified: 3 Apr 2021 | 11:23:34 UTC

You don't have to do anything if you don't want to, it still runs just slowly. I just figure given how much running 24x7 costs in electricity you'd want to get maximum bang for your buck.

I do, within reason. But presumably it's a hassle (and even impossible?) to set different amounts of cores for each subproject, so I'd have to do the same for all of them. I've had a look for the 24 core machines, the other machines only have 4 and 6 cores so I take it those don't need fiddling with.

It does also provide heating for my lovebird aviary. But when it's not winter, the window gets opened, so I guess it's still allowing them more ventilation.

But, this does not address my original post. Why is Primegrid handing out WAY too much work? With a 3 hour buffer I do not expect a week to be downloaded. The adjustments you've suggested are not to a factor of 24 or even 50, so it's not that. This is a server misconfiguration or bug which should be fixed. Until then I have to manually abort a lot of tasks that will never complete on time (I do other projects too).

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149824 - Posted: 3 Apr 2021 | 11:20:53 UTC
Last modified: 3 Apr 2021 | 11:29:44 UTC

I tried Prime95 on my Ryzen and one of my xeons, and the 6 core i5.
On the 12 core 24 thread Ryzen it's substantially faster using 4, 6, or 8 workers.
On the dual CPU 2 of 6 core 12 thread Xeon it's substantially faster using 2, 3, or 4 workers.
On the 6 core no HT i5 it's faster using 1 worker.

So it looks like it's always best to use 6 cores per task, so I've set in the server prefs to use 6 cores per task, does that make sense?
I'm just not sure about the hyperthreading. I'm leaving that on and I'm not sure that Prime95 is taking account of it, or telling me the most sensible number of workers to use.

Profile j.sheridanProject donor
Volunteer tester
Send message
Joined: 21 Mar 11
Posts: 816
ID: 91622
Credit: 2,331,853,674
RAC: 2,280,044
Discovered 2 mega primesFound 4 primes in the 2021 Tour de PrimesFound 2 mega primes in the 2021 Tour de Primes321 LLR Jade: Earned 10,000,000 credits (10,011,712)Cullen LLR Jade: Earned 10,000,000 credits (10,025,728)ESP LLR Jade: Earned 10,000,000 credits (10,176,490)Generalized Cullen/Woodall LLR Jade: Earned 10,000,000 credits (10,020,580)PPS LLR Sapphire: Earned 20,000,000 credits (29,059,451)PSP LLR Jade: Earned 10,000,000 credits (10,055,994)SoB LLR Sapphire: Earned 20,000,000 credits (39,949,623)SR5 LLR Jade: Earned 10,000,000 credits (10,885,368)SGS LLR Jade: Earned 10,000,000 credits (10,007,143)TRP LLR Jade: Earned 10,000,000 credits (10,009,531)Woodall LLR Jade: Earned 10,000,000 credits (10,039,527)321 Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,019,388)Cullen/Woodall Sieve (suspended) Double Silver: Earned 200,000,000 credits (265,102,350)PPS Sieve Double Gold: Earned 500,000,000 credits (547,992,812)TRP Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,801,812)AP 26/27 Emerald: Earned 50,000,000 credits (93,546,934)GFN Double Silver: Earned 200,000,000 credits (436,635,654)WW Double Gold: Earned 500,000,000 credits (802,548,000)
Message 149826 - Posted: 3 Apr 2021 | 15:34:04 UTC - in response to Message 149824.

If you don't want to mess around adjusting things per subproject then 6 threads would be the best choice on the ryzen. You also need to set "use at most: 50% processors" in your boinc options.

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149830 - Posted: 3 Apr 2021 | 17:36:59 UTC - in response to Message 149826.
Last modified: 3 Apr 2021 | 17:37:31 UTC

If you don't want to mess around adjusting things per subproject then 6 threads would be the best choice on the ryzen. You also need to set "use at most: 50% processors" in your boinc options.

Yes, I've set 6 threads globally on the server preferences.

Why would I want to set 50% processors? HT speeds things up as a rule. I've confirmed this by timing tasks from various projects.

Michael Millerick
Volunteer tester
Send message
Joined: 4 Feb 09
Posts: 799
ID: 35074
Credit: 319,977,601
RAC: 313,520
Discovered 1 mega primeFound 3 primes in the 2020 Tour de PrimesFound 2 primes in the 2021 Tour de Primes321 LLR Ruby: Earned 2,000,000 credits (2,023,600)Cullen LLR Ruby: Earned 2,000,000 credits (2,016,794)ESP LLR Sapphire: Earned 20,000,000 credits (20,186,009)Generalized Cullen/Woodall LLR Ruby: Earned 2,000,000 credits (2,614,647)PPS LLR Jade: Earned 10,000,000 credits (19,232,686)PSP LLR Ruby: Earned 2,000,000 credits (3,043,852)SoB LLR Ruby: Earned 2,000,000 credits (2,031,453)SR5 LLR Ruby: Earned 2,000,000 credits (2,085,475)SGS LLR Ruby: Earned 2,000,000 credits (2,035,177)TRP LLR Ruby: Earned 2,000,000 credits (2,740,707)Woodall LLR Ruby: Earned 2,000,000 credits (2,046,592)321 Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,440,175)Cullen/Woodall Sieve (suspended) Ruby: Earned 2,000,000 credits (4,911,344)Generalized Cullen/Woodall Sieve (suspended) Ruby: Earned 2,000,000 credits (2,077,092)PPS Sieve Emerald: Earned 50,000,000 credits (50,200,870)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Amethyst: Earned 1,000,000 credits (1,034,014)TRP Sieve (suspended) Ruby: Earned 2,000,000 credits (2,070,774)AP 26/27 Sapphire: Earned 20,000,000 credits (20,227,947)GFN Emerald: Earned 50,000,000 credits (70,992,100)WW Emerald: Earned 50,000,000 credits (50,104,000)PSA Emerald: Earned 50,000,000 credits (52,866,806)
Message 149831 - Posted: 3 Apr 2021 | 17:47:17 UTC - in response to Message 149830.

If you don't want to mess around adjusting things per subproject then 6 threads would be the best choice on the ryzen. You also need to set "use at most: 50% processors" in your boinc options.

Yes, I've set 6 threads globally on the server preferences.

Why would I want to set 50% processors? HT speeds things up as a rule. I've confirmed this by timing tasks from various projects.

That is not the case with the AVX workloads of LLR.
____________

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149832 - Posted: 3 Apr 2021 | 17:57:19 UTC - in response to Message 149831.
Last modified: 3 Apr 2021 | 17:57:48 UTC

If you don't want to mess around adjusting things per subproject then 6 threads would be the best choice on the ryzen. You also need to set "use at most: 50% processors" in your boinc options.

Yes, I've set 6 threads globally on the server preferences.

Why would I want to set 50% processors? HT speeds things up as a rule. I've confirmed this by timing tasks from various projects.

That is not the case with the AVX workloads of LLR.

Unfortunately I don't think I can tell Boinc to change HT use per project. Even if there is such a function, when running some tasks from two different projects at once, things would become complicated.

Since I'm leaving HT on, is 6 threads per task still the best choice?

Profile j.sheridanProject donor
Volunteer tester
Send message
Joined: 21 Mar 11
Posts: 816
ID: 91622
Credit: 2,331,853,674
RAC: 2,280,044
Discovered 2 mega primesFound 4 primes in the 2021 Tour de PrimesFound 2 mega primes in the 2021 Tour de Primes321 LLR Jade: Earned 10,000,000 credits (10,011,712)Cullen LLR Jade: Earned 10,000,000 credits (10,025,728)ESP LLR Jade: Earned 10,000,000 credits (10,176,490)Generalized Cullen/Woodall LLR Jade: Earned 10,000,000 credits (10,020,580)PPS LLR Sapphire: Earned 20,000,000 credits (29,059,451)PSP LLR Jade: Earned 10,000,000 credits (10,055,994)SoB LLR Sapphire: Earned 20,000,000 credits (39,949,623)SR5 LLR Jade: Earned 10,000,000 credits (10,885,368)SGS LLR Jade: Earned 10,000,000 credits (10,007,143)TRP LLR Jade: Earned 10,000,000 credits (10,009,531)Woodall LLR Jade: Earned 10,000,000 credits (10,039,527)321 Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,019,388)Cullen/Woodall Sieve (suspended) Double Silver: Earned 200,000,000 credits (265,102,350)PPS Sieve Double Gold: Earned 500,000,000 credits (547,992,812)TRP Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,801,812)AP 26/27 Emerald: Earned 50,000,000 credits (93,546,934)GFN Double Silver: Earned 200,000,000 credits (436,635,654)WW Double Gold: Earned 500,000,000 credits (802,548,000)
Message 149833 - Posted: 3 Apr 2021 | 18:01:46 UTC - in response to Message 149832.
Last modified: 3 Apr 2021 | 18:02:28 UTC

If you set 50% processors you're effectively turning HT off for primegrid. If you don't want to do that you can also set maximum tasks to 2.

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149834 - Posted: 3 Apr 2021 | 18:08:24 UTC - in response to Message 149833.
Last modified: 3 Apr 2021 | 18:08:33 UTC

If you set 50% processors you're effectively turning HT off for primegrid. If you don't want to do that you can also set maximum tasks to 2.

But Boinc will still run other projects on the other threads, so HT is on again.

Michael Millerick
Volunteer tester
Send message
Joined: 4 Feb 09
Posts: 799
ID: 35074
Credit: 319,977,601
RAC: 313,520
Discovered 1 mega primeFound 3 primes in the 2020 Tour de PrimesFound 2 primes in the 2021 Tour de Primes321 LLR Ruby: Earned 2,000,000 credits (2,023,600)Cullen LLR Ruby: Earned 2,000,000 credits (2,016,794)ESP LLR Sapphire: Earned 20,000,000 credits (20,186,009)Generalized Cullen/Woodall LLR Ruby: Earned 2,000,000 credits (2,614,647)PPS LLR Jade: Earned 10,000,000 credits (19,232,686)PSP LLR Ruby: Earned 2,000,000 credits (3,043,852)SoB LLR Ruby: Earned 2,000,000 credits (2,031,453)SR5 LLR Ruby: Earned 2,000,000 credits (2,085,475)SGS LLR Ruby: Earned 2,000,000 credits (2,035,177)TRP LLR Ruby: Earned 2,000,000 credits (2,740,707)Woodall LLR Ruby: Earned 2,000,000 credits (2,046,592)321 Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,440,175)Cullen/Woodall Sieve (suspended) Ruby: Earned 2,000,000 credits (4,911,344)Generalized Cullen/Woodall Sieve (suspended) Ruby: Earned 2,000,000 credits (2,077,092)PPS Sieve Emerald: Earned 50,000,000 credits (50,200,870)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Amethyst: Earned 1,000,000 credits (1,034,014)TRP Sieve (suspended) Ruby: Earned 2,000,000 credits (2,070,774)AP 26/27 Sapphire: Earned 20,000,000 credits (20,227,947)GFN Emerald: Earned 50,000,000 credits (70,992,100)WW Emerald: Earned 50,000,000 credits (50,104,000)PSA Emerald: Earned 50,000,000 credits (52,866,806)
Message 149836 - Posted: 3 Apr 2021 | 23:26:31 UTC - in response to Message 149834.

If you set 50% processors you're effectively turning HT off for primegrid. If you don't want to do that you can also set maximum tasks to 2.

But Boinc will still run other projects on the other threads, so HT is on again.

I think that is the difference here. I think that most of us that have been responding to you don't run any other Boinc projects while we are also running PrimeGrid. You may want to restrict yourself to the non-LLR projects if you don't want to disable other projects, but then you also won't be able to find a Prime, or just let your computer chug along. For all of the LLR projects except for PPSE and SGS, it doesn't actually matter how fast you return the task now, you will still get the credit for finding a prime since there is no longer a race with the double checker.
____________

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149849 - Posted: 4 Apr 2021 | 17:46:02 UTC - in response to Message 149836.

If you set 50% processors you're effectively turning HT off for primegrid. If you don't want to do that you can also set maximum tasks to 2.

But Boinc will still run other projects on the other threads, so HT is on again.

I think that is the difference here. I think that most of us that have been responding to you don't run any other Boinc projects while we are also running PrimeGrid. You may want to restrict yourself to the non-LLR projects if you don't want to disable other projects, but then you also won't be able to find a Prime, or just let your computer chug along. For all of the LLR projects except for PPSE and SGS, it doesn't actually matter how fast you return the task now, you will still get the credit for finding a prime since there is no longer a race with the double checker.

I'm not convinced on HT being that bad. All I've seen bad about it is it causes your cache to be used up, so accesses RAM instead, which is slower. Two things wrong with that:

1) Surely running more tasks at once and reducing the threads per task has the same effect?

2) I built this Ryzen with single channel RAM, as I was told most games aren't any faster with it. When I read another article that the game I play (Fallout 4) IS faster with dual channel, I bought another stick. It wasn't faster (which could be my slow GPU's fault), but it also wasn't faster with any Boinc project including this one. Now if the accessing RAM was causing a slowdown, then doubling the speed of the RAM by adding dual channel would have halved that slowness. It didn't.

Profile JeppeSNProject donor
Avatar
Send message
Joined: 5 Apr 14
Posts: 1533
ID: 306875
Credit: 35,605,759
RAC: 10,132
Found 1 prime in the 2020 Tour de Primes321 LLR Gold: Earned 500,000 credits (529,293)Cullen LLR Gold: Earned 500,000 credits (611,298)ESP LLR Silver: Earned 100,000 credits (174,818)Generalized Cullen/Woodall LLR Bronze: Earned 10,000 credits (35,236)PPS LLR Jade: Earned 10,000,000 credits (13,139,146)PSP LLR Silver: Earned 100,000 credits (428,457)SoB LLR Silver: Earned 100,000 credits (466,812)SR5 LLR Silver: Earned 100,000 credits (145,419)SGS LLR Silver: Earned 100,000 credits (112,277)TRP LLR Silver: Earned 100,000 credits (342,501)Woodall LLR Silver: Earned 100,000 credits (109,455)321 Sieve (suspended) Silver: Earned 100,000 credits (175,037)PPS Sieve Bronze: Earned 10,000 credits (10,113)AP 26/27 Bronze: Earned 10,000 credits (12,129)GFN Ruby: Earned 2,000,000 credits (2,059,478)WW Turquoise: Earned 5,000,000 credits (9,640,000)PSA Turquoise: Earned 5,000,000 credits (7,614,290)
Message 149850 - Posted: 4 Apr 2021 | 18:14:42 UTC - in response to Message 149849.

Peter Hucker, when all participants unanimously say that hyperthreading (HT) is bad for you when you crunch PrimeGrid, and that you must either turn off HT in the BIOS, or ensure that PrimeGrid uses only 50% of the "processors" (meaning 50% of the logical processors), then you should believe them!

If you run on all logical threads (HT), the speed of each process will be less than half the normal value, and as a result, the throughput will be smaller than when you run it the way everybody recommends.

Hyperthreading works well for most typical software, programs and apps. That is because the software most people use on their computers, does very complex tasks, and is programmed on a high level, and as a result, does not put heavy pressure on the CPU and memory. But PrimeGrid's apps, on the other hand, do the same type of "simple" (mathematical) calculation (i.e. the same CPU instructions) over and over again, and the apps are very highly optimized to squeezing the most out of the hardware. There is a difference between having two independent, "real" CPU cores, versus having two hyperthreads running on one physical core. For "normal" software, that does not resemble PrimeGrid's, it is not so big a problem to have to share some ressource with another thread. But for PrimeGrid's apps (perhaps except for some sieving apps), two hyperthreads on the same physical core will just be fighting each other to get access to a ressource of which there is only one, and the overhead of shifting from one to the other, and invalidating cache from the opposite thread, etc. (someone can teach me the precise details), will make things slower than without HT.

But if you do not believe it, it is easy to measure yourself.

/JeppeSN

Profile NickProject donor
Avatar
Send message
Joined: 11 Jul 11
Posts: 999
ID: 105020
Credit: 2,701,352,140
RAC: 8,969,123
Discovered 8 mega primesFound 5 primes in the 2020 Tour de PrimesFound 2 mega primes in the 2020 Tour de PrimesFound 4 primes in the 2021 Tour de PrimesFound 2 mega primes in the 2021 Tour de Primes321 LLR Sapphire: Earned 20,000,000 credits (20,020,701)Cullen LLR Sapphire: Earned 20,000,000 credits (20,174,451)ESP LLR Sapphire: Earned 20,000,000 credits (23,496,803)Generalized Cullen/Woodall LLR Sapphire: Earned 20,000,000 credits (20,197,125)PPS LLR Emerald: Earned 50,000,000 credits (61,332,778)PSP LLR Emerald: Earned 50,000,000 credits (51,069,305)SoB LLR Sapphire: Earned 20,000,000 credits (32,363,321)SR5 LLR Sapphire: Earned 20,000,000 credits (20,183,763)SGS LLR Sapphire: Earned 20,000,000 credits (20,482,069)TRP LLR Sapphire: Earned 20,000,000 credits (20,307,419)Woodall LLR Sapphire: Earned 20,000,000 credits (22,371,625)321 Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,380,527)Cullen/Woodall Sieve (suspended) Gold: Earned 500,000 credits (744,531)Generalized Cullen/Woodall Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,007,004)PPS Sieve Double Gold: Earned 500,000,000 credits (575,268,110)TRP Sieve (suspended) Bronze: Earned 10,000 credits (21,181)AP 26/27 Sapphire: Earned 20,000,000 credits (20,845,708)GFN Double Silver: Earned 200,000,000 credits (482,041,720)WW Double Amethyst: Earned 1,000,000,000 credits (1,285,368,000)
Message 149851 - Posted: 4 Apr 2021 | 19:51:12 UTC - in response to Message 149850.

(someone can teach me the precise details)

I am also worried about saying things well and precisely. If JeppeSN has this concern, I think PG message is too worried about details. But yes, we should attempt to be factually correct. I also think there should be an attempt to think best of what people write - so we don't have to be so concerned for absolute correctness. I wonder at the meaning of The Devil In The Details....

Profile robishProject donor
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 7 Jan 12
Posts: 1993
ID: 126266
Credit: 5,863,173,310
RAC: 2,567,932
Discovered the World's First AP27!!!Discovered 9 mega primesDiscovered 1 AP272018 Tour de Primes largest primeFound 4 primes in the 2018 Tour de PrimesFound 1 mega prime in the 2018 Tour de PrimesFound 1 prime in the 2019 Tour de PrimesFound 1 prime in the 2020 Tour de PrimesFound 1 prime in the 2021 Tour de Primes321 LLR Turquoise: Earned 5,000,000 credits (9,681,253)Cullen LLR Sapphire: Earned 20,000,000 credits (46,627,058)ESP LLR Turquoise: Earned 5,000,000 credits (8,577,288)Generalized Cullen/Woodall LLR Turquoise: Earned 5,000,000 credits (9,950,938)PPS LLR Emerald: Earned 50,000,000 credits (81,462,474)PSP LLR Turquoise: Earned 5,000,000 credits (9,600,575)SoB LLR Sapphire: Earned 20,000,000 credits (36,846,875)SR5 LLR Turquoise: Earned 5,000,000 credits (9,986,029)SGS LLR Turquoise: Earned 5,000,000 credits (5,658,541)TRP LLR Sapphire: Earned 20,000,000 credits (28,889,840)Woodall LLR Turquoise: Earned 5,000,000 credits (5,062,771)321 Sieve (suspended) Turquoise: Earned 5,000,000 credits (7,141,753)Cullen/Woodall Sieve (suspended) Turquoise: Earned 5,000,000 credits (7,892,369)Generalized Cullen/Woodall Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,515,338)PPS Sieve Double Gold: Earned 500,000,000 credits (842,736,516)TRP Sieve (suspended) Silver: Earned 100,000 credits (121,416)AP 26/27 Emerald: Earned 50,000,000 credits (93,931,019)GFN Double Ruby: Earned 2,000,000,000 credits (4,557,436,895)WW Emerald: Earned 50,000,000 credits (96,052,000)
Message 149852 - Posted: 5 Apr 2021 | 0:40:39 UTC
Last modified: 5 Apr 2021 | 7:18:52 UTC

There are hundreds, no thousands of PG participatants who have experience testing and retesting the optimal setup for the software. Only one is needed to share this experience so others dont have to do it themselves.
Trust is an issue I personally didn't consider as a problem when I begun here. I assumed everyone else knew more than me and I was correct.

This project and its software are somewhat unique in the way they're optimised.
People give the best answer to a question, not from personal opinion but from collective knowledge. So as not to reinvent the wheel, repeatedly.
Its probably a good idea to accept the free advice and worry about the reason later. It becomes clear after time.
We are all learning.

Edit: joke removed
____________
My lucky numbers 10590941048576+1 and 224584605939537911+81292139*23#*n for n=0..26

Profile Eudy SilvaProject donor
Avatar
Send message
Joined: 26 Aug 17
Posts: 1538
ID: 918937
Credit: 340,879,241
RAC: 188,844
Discovered 2 mega primesFound 2 primes in the 2018 Tour de PrimesFound 1 prime in the 2019 Tour de PrimesFound 1 prime in the 2020 Tour de PrimesFound 1 prime in the 2021 Tour de Primes321 LLR Amethyst: Earned 1,000,000 credits (1,012,503)Cullen LLR Amethyst: Earned 1,000,000 credits (1,291,013)ESP LLR Amethyst: Earned 1,000,000 credits (1,151,976)Generalized Cullen/Woodall LLR Amethyst: Earned 1,000,000 credits (1,004,746)PPS LLR Jade: Earned 10,000,000 credits (19,858,774)PSP LLR Amethyst: Earned 1,000,000 credits (1,476,639)SoB LLR Ruby: Earned 2,000,000 credits (2,432,091)SR5 LLR Ruby: Earned 2,000,000 credits (2,954,518)SGS LLR Amethyst: Earned 1,000,000 credits (1,462,951)TRP LLR Amethyst: Earned 1,000,000 credits (1,574,163)Woodall LLR Amethyst: Earned 1,000,000 credits (1,001,901)321 Sieve (suspended) Amethyst: Earned 1,000,000 credits (1,119,749)Generalized Cullen/Woodall Sieve (suspended) Amethyst: Earned 1,000,000 credits (1,005,331)PPS Sieve Emerald: Earned 50,000,000 credits (68,690,867)AP 26/27 Emerald: Earned 50,000,000 credits (50,096,813)GFN Emerald: Earned 50,000,000 credits (78,674,887)WW Double Bronze: Earned 100,000,000 credits (104,012,000)PSA Ruby: Earned 2,000,000 credits (2,059,222)
Message 149853 - Posted: 5 Apr 2021 | 2:34:17 UTC - in response to Message 149850.

JeppeSN wrote:
... or ensure that PrimeGrid uses only 50% of the "processors" (meaning 50% of the logical processors) ...

JeppeSN, please correct me if I'm wrong here, I have the impression that "use at most 50% of the CPUs" setting only affects BOINC's calculation of the quantity of tasks it will ask for download from the server.
The actual distribution of work load among the cores is managed by the OS (with the optional help of third party software, like Process Lasso for Windows).

____________
"Accidit in puncto, quod non contingit in anno."
Something that does not occur in a year may, perchance, happen in a moment.

Profile dannyridel
Volunteer tester
Avatar
Send message
Joined: 3 Feb 19
Posts: 919
ID: 1097922
Credit: 20,030,100
RAC: 24,214
Discovered 1 mega prime321 LLR Gold: Earned 500,000 credits (506,806)Cullen LLR Silver: Earned 100,000 credits (112,212)ESP LLR Silver: Earned 100,000 credits (255,803)Generalized Cullen/Woodall LLR Gold: Earned 500,000 credits (509,419)PPS LLR Ruby: Earned 2,000,000 credits (2,001,172)PSP LLR Gold: Earned 500,000 credits (616,940)SoB LLR Gold: Earned 500,000 credits (533,625)SR5 LLR Silver: Earned 100,000 credits (108,281)SGS LLR Gold: Earned 500,000 credits (501,474)TRP LLR Silver: Earned 100,000 credits (146,999)Woodall LLR Silver: Earned 100,000 credits (117,864)321 Sieve (suspended) Gold: Earned 500,000 credits (506,814)Generalized Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (30,033)PPS Sieve Ruby: Earned 2,000,000 credits (2,808,043)AP 26/27 Amethyst: Earned 1,000,000 credits (1,273,545)GFN Ruby: Earned 2,000,000 credits (3,480,035)WW Turquoise: Earned 5,000,000 credits (6,148,000)PSA Silver: Earned 100,000 credits (373,034)
Message 149856 - Posted: 5 Apr 2021 | 4:44:04 UTC - in response to Message 149853.

JeppeSN wrote:
... or ensure that PrimeGrid uses only 50% of the "processors" (meaning 50% of the logical processors) ...

JeppeSN, please correct me if I'm wrong here, I have the impression that "use at most 50% of the CPUs" setting only affects BOINC's calculation of the quantity of tasks it will ask for download from the server.
The actual distribution of work load among the cores is managed by the OS (with the optional help of third party software, like Process Lasso for Windows).

Technically smarter OS's like Linux will just adjust the threads to one per actual core. However most of the time even Windows can do this kind of thing correctly.
To be sure the OS behaves, affinitywatcher can be used in this case.
____________
SHSIDElectronicsGroup@outlook.com

waiting for a TdP prime...
Proth "SoB": 44243*2^440969+1


Profile composite
Volunteer tester
Send message
Joined: 16 Feb 10
Posts: 841
ID: 55391
Credit: 782,673,526
RAC: 442,633
Discovered 2 mega primesFound 1 prime in the 2018 Tour de Primes321 LLR Turquoise: Earned 5,000,000 credits (5,477,467)Cullen LLR Gold: Earned 500,000 credits (776,297)ESP LLR Ruby: Earned 2,000,000 credits (3,433,680)Generalized Cullen/Woodall LLR Ruby: Earned 2,000,000 credits (2,093,491)PPS LLR Sapphire: Earned 20,000,000 credits (27,299,451)PSP LLR Turquoise: Earned 5,000,000 credits (6,587,988)SoB LLR Sapphire: Earned 20,000,000 credits (33,929,863)SR5 LLR Turquoise: Earned 5,000,000 credits (6,110,877)SGS LLR Ruby: Earned 2,000,000 credits (3,486,285)TRP LLR Turquoise: Earned 5,000,000 credits (7,070,795)Woodall LLR Amethyst: Earned 1,000,000 credits (1,693,614)321 Sieve (suspended) Emerald: Earned 50,000,000 credits (50,256,050)Cullen/Woodall Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,571,178)Generalized Cullen/Woodall Sieve (suspended) Emerald: Earned 50,000,000 credits (50,009,610)PPS Sieve Double Silver: Earned 200,000,000 credits (384,557,559)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Jade: Earned 10,000,000 credits (10,165,888)TRP Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,071,454)AP 26/27 Turquoise: Earned 5,000,000 credits (6,616,128)GFN Emerald: Earned 50,000,000 credits (53,771,465)WW Gold: Earned 500,000 credits (932,000)PSA Double Bronze: Earned 100,000,000 credits (102,762,384)
Message 149857 - Posted: 5 Apr 2021 | 5:18:23 UTC - in response to Message 149849.
Last modified: 5 Apr 2021 | 5:26:40 UTC

Peter Hucker wrote:
I bought another stick. It wasn't faster [...] Now if the accessing RAM was causing a slowdown, then doubling the speed of the RAM by adding dual channel would have halved that slowness. It didn't.

1) Not every memory access goes to RAM. The cache is still is still there cutting out some RAM access, so improvement will be automatically less than you expect.

2) Dual channel doesn't double the speed of the RAM, it doubles the data path. Typical speed improvement is much less than 100% (double); 10% to 15% according to what I read, and 20% to 50% (for DDR3) according to this article at Tom's Hardware. Sure, the bandwidth has doubled, but not all of the RAM's access time is about moving data across the bus. There's latency between supplying the memory address and receiving the data. Also, the data doesn't come out of any individual channel any faster so if your RAM access isn't interleaved between the channels, you will have zero speed improvement.

3) It's unusual to go from single channel RAM to dual channel RAM by adding a single stick due to the difficulties in getting matched memory sticks. RAM is usually sold as a multi-channel package because the vendor ensures the RAM timing is the same across all sticks. This kind of upgrade might even cause your computer to go slower without fine-tuning. See the aformentioned article about this bugaboo.

Profile JeppeSNProject donor
Avatar
Send message
Joined: 5 Apr 14
Posts: 1533
ID: 306875
Credit: 35,605,759
RAC: 10,132
Found 1 prime in the 2020 Tour de Primes321 LLR Gold: Earned 500,000 credits (529,293)Cullen LLR Gold: Earned 500,000 credits (611,298)ESP LLR Silver: Earned 100,000 credits (174,818)Generalized Cullen/Woodall LLR Bronze: Earned 10,000 credits (35,236)PPS LLR Jade: Earned 10,000,000 credits (13,139,146)PSP LLR Silver: Earned 100,000 credits (428,457)SoB LLR Silver: Earned 100,000 credits (466,812)SR5 LLR Silver: Earned 100,000 credits (145,419)SGS LLR Silver: Earned 100,000 credits (112,277)TRP LLR Silver: Earned 100,000 credits (342,501)Woodall LLR Silver: Earned 100,000 credits (109,455)321 Sieve (suspended) Silver: Earned 100,000 credits (175,037)PPS Sieve Bronze: Earned 10,000 credits (10,113)AP 26/27 Bronze: Earned 10,000 credits (12,129)GFN Ruby: Earned 2,000,000 credits (2,059,478)WW Turquoise: Earned 5,000,000 credits (9,640,000)PSA Turquoise: Earned 5,000,000 credits (7,614,290)
Message 149860 - Posted: 5 Apr 2021 | 8:02:43 UTC - in response to Message 149853.

JeppeSN wrote:
... or ensure that PrimeGrid uses only 50% of the "processors" (meaning 50% of the logical processors) ...

JeppeSN, please correct me if I'm wrong here, I have the impression that "use at most 50% of the CPUs" setting only affects BOINC's calculation of the quantity of tasks it will ask for download from the server.
The actual distribution of work load among the cores is managed by the OS (with the optional help of third party software, like Process Lasso for Windows).

It should affect the amount of tasks BOINC will process at the same time (no matter if it has a buffer of additional, waiting tasks or not). You are absolutely correct the OS (operating system, like Windows or Linux) will distribute the work load. Hopefully, most OSs will know to put the different tasks onto separate physical cores (as long as the number of processes (where the OS must be able to see which ones are relevant) is small enough to allow it). I heard even Windows has become better at this.

But as you say, you can control/overwrite this, or just observe it, with a tool.

/JeppeSN

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149874 - Posted: 6 Apr 2021 | 17:03:49 UTC - in response to Message 149850.

Peter Hucker, when all participants unanimously say that hyperthreading (HT) is bad for you when you crunch PrimeGrid, and that you must either turn off HT in the BIOS, or ensure that PrimeGrid uses only 50% of the "processors" (meaning 50% of the logical processors), then you should believe them!

I never said they were lying, it just sounds weird. And that I'm not prepared to adjust things which only improve one project and make all the others slower. I'm also not convinced it's true on all processors. AMD and Intel for example have very different amounts of cache etc.

If Primegrid is so well written compared to other programs, could it not include something to deliberately not use HT? For example it could detect how many cores there are, how many Boinc tasks are running, and if you haven't already adjusted yourself as already mentioned, it could run less processes - taking up two threads in the Boinc scheduler but only using one.


Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149875 - Posted: 6 Apr 2021 | 17:04:38 UTC - in response to Message 149852.

There are hundreds, no thousands of PG participatants who have experience testing and retesting the optimal setup for the software. Only one is needed to share this experience so others dont have to do it themselves.
Trust is an issue I personally didn't consider as a problem when I begun here. I assumed everyone else knew more than me and I was correct.

This project and its software are somewhat unique in the way they're optimised.
People give the best answer to a question, not from personal opinion but from collective knowledge. So as not to reinvent the wheel, repeatedly.
Its probably a good idea to accept the free advice and worry about the reason later. It becomes clear after time.
We are all learning.

Edit: joke removed

You can't do that! Now we all want to know what it was!

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149876 - Posted: 6 Apr 2021 | 17:08:44 UTC - in response to Message 149857.

Peter Hucker wrote:
I bought another stick. It wasn't faster [...] Now if the accessing RAM was causing a slowdown, then doubling the speed of the RAM by adding dual channel would have halved that slowness. It didn't.

1) Not every memory access goes to RAM. The cache is still is still there cutting out some RAM access, so improvement will be automatically less than you expect.
I guess there's also the question of which of the 3 levels of cache are being used.

2) Dual channel doesn't double the speed of the RAM, it doubles the data path. Typical speed improvement is much less than 100% (double); 10% to 15% according to what I read, and 20% to 50% (for DDR3) according to this article at Tom's Hardware. Sure, the bandwidth has doubled, but not all of the RAM's access time is about moving data across the bus. There's latency between supplying the memory address and receiving the data. Also, the data doesn't come out of any individual channel any faster so if your RAM access isn't interleaved between the channels, you will have zero speed improvement.
Why do they use it if it's only 10-15%? In fact I didn't even observe that much increase, but I think my CPU has a large cache.

3) It's unusual to go from single channel RAM to dual channel RAM by adding a single stick due to the difficulties in getting matched memory sticks. RAM is usually sold as a multi-channel package because the vendor ensures the RAM timing is the same across all sticks. This kind of upgrade might even cause your computer to go slower without fine-tuning. See the aformentioned article about this bugaboo.
Not sure why people always say that. You just buy any stick with the same CAS. Even if you don't, the BIOS/MB just runs them at the lesser of the two speeds. I've got some machines with totally mismatched RAM, they just run as though I had two of the slower ones. Same goes for striping hard disks in a RAID.

Profile composite
Volunteer tester
Send message
Joined: 16 Feb 10
Posts: 841
ID: 55391
Credit: 782,673,526
RAC: 442,633
Discovered 2 mega primesFound 1 prime in the 2018 Tour de Primes321 LLR Turquoise: Earned 5,000,000 credits (5,477,467)Cullen LLR Gold: Earned 500,000 credits (776,297)ESP LLR Ruby: Earned 2,000,000 credits (3,433,680)Generalized Cullen/Woodall LLR Ruby: Earned 2,000,000 credits (2,093,491)PPS LLR Sapphire: Earned 20,000,000 credits (27,299,451)PSP LLR Turquoise: Earned 5,000,000 credits (6,587,988)SoB LLR Sapphire: Earned 20,000,000 credits (33,929,863)SR5 LLR Turquoise: Earned 5,000,000 credits (6,110,877)SGS LLR Ruby: Earned 2,000,000 credits (3,486,285)TRP LLR Turquoise: Earned 5,000,000 credits (7,070,795)Woodall LLR Amethyst: Earned 1,000,000 credits (1,693,614)321 Sieve (suspended) Emerald: Earned 50,000,000 credits (50,256,050)Cullen/Woodall Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,571,178)Generalized Cullen/Woodall Sieve (suspended) Emerald: Earned 50,000,000 credits (50,009,610)PPS Sieve Double Silver: Earned 200,000,000 credits (384,557,559)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Jade: Earned 10,000,000 credits (10,165,888)TRP Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,071,454)AP 26/27 Turquoise: Earned 5,000,000 credits (6,616,128)GFN Emerald: Earned 50,000,000 credits (53,771,465)WW Gold: Earned 500,000 credits (932,000)PSA Double Bronze: Earned 100,000,000 credits (102,762,384)
Message 149889 - Posted: 7 Apr 2021 | 5:17:50 UTC - in response to Message 149876.


3) It's unusual to go from single channel RAM to dual channel RAM by adding a single stick due to the difficulties in getting matched memory sticks. RAM is usually sold as a multi-channel package because the vendor ensures the RAM timing is the same across all sticks. This kind of upgrade might even cause your computer to go slower without fine-tuning. See the aformentioned article about this bugaboo.
Not sure why people always say that. You just buy any stick with the same CAS. Even if you don't, the BIOS/MB just runs them at the lesser of the two speeds. I've got some machines with totally mismatched RAM, they just run as though I had two of the slower ones. Same goes for striping hard disks in a RAID.

I am inclined to agree with you on theoretical grounds but in the general case the advice has to be conservative when not inserting caveats and disclaimers everywhere. That's a slightly aged article so the author is probably referring to an older BIOS behaviour - just ignoring the XMP settings altogether and dropping the RAM clock to the minimum JEDEC spec just to make sure the system is bootable. I have a 7 year old X99 mobo which does exactly that even with a proper matched set of RAM if it doesn't like what I've done with a custom RAM setting.

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149946 - Posted: 9 Apr 2021 | 13:24:15 UTC - in response to Message 149852.
Last modified: 9 Apr 2021 | 13:38:07 UTC

There are hundreds, no thousands of PG participatants who have experience testing and retesting the optimal setup for the software. Only one is needed to share this experience so others dont have to do it themselves.
Trust is an issue I personally didn't consider as a problem when I begun here. I assumed everyone else knew more than me and I was correct.

This project and its software are somewhat unique in the way they're optimised.
People give the best answer to a question, not from personal opinion but from collective knowledge. So as not to reinvent the wheel, repeatedly.
Its probably a good idea to accept the free advice and worry about the reason later. It becomes clear after time.
We are all learning.

Test results for Ryzen 9 3900XT (12 core 24 thread):

Primegrid "Prime Sierpinski Problem (LLR) (mt)" using 6 threads per task:
Turning on HT runs them at 1/3 speed, but twice as many so total throughput is 2/3.

Private GFN Server "GFN-13 MEGA Prime Search" using 3 threads per task:
Turning on HT runs them at 2/3 speed, but twice as many so total throughput is 4/3.

Universe "Universe BH Spin v2" single thread tasks:
Turning on HT runs them at the same speed, but twice as many so total throughput is times 2.

So for my sort of CPU, only Primegrid (and not even the test server) benefits from HT being turned off. So it depends what you're running. I run a mix of projects, so I'll leave it on.

This does beg the question though, does this mean other projects have sloppy programming compared to Primegrid? Could their programs be fine tuned? Or is it just that Primegrid by it's nature does more RAM access and floods the cache if HT is used? Climate prediction forums discussed a shortage of cache in a similar way. They're large data sets, so it could be the same thing as here.

Extra test - power consumption.
HT off: 2 of 6 thread Primegrid "Prime Sierpinski Problem (LLR) (mt)" uses 141W.
HT on: 4 of 6 thread Primegrid "Prime Sierpinski Problem (LLR) (mt)" uses 118W.
So HT gives you 2/3 of the work done, but uses 83.5% of the power.

Profile Rafael
Volunteer tester
Avatar
Send message
Joined: 22 Oct 14
Posts: 899
ID: 370496
Credit: 411,941,744
RAC: 209,508
321 LLR Turquoise: Earned 5,000,000 credits (8,236,942)Cullen LLR Turquoise: Earned 5,000,000 credits (8,028,695)ESP LLR Turquoise: Earned 5,000,000 credits (8,437,609)Generalized Cullen/Woodall LLR Turquoise: Earned 5,000,000 credits (8,011,297)PPS LLR Turquoise: Earned 5,000,000 credits (8,392,935)PSP LLR Turquoise: Earned 5,000,000 credits (9,998,922)SoB LLR Turquoise: Earned 5,000,000 credits (9,310,410)SR5 LLR Turquoise: Earned 5,000,000 credits (9,959,965)SGS LLR Turquoise: Earned 5,000,000 credits (8,944,665)TRP LLR Turquoise: Earned 5,000,000 credits (8,751,841)Woodall LLR Turquoise: Earned 5,000,000 credits (9,876,482)321 Sieve (suspended) Jade: Earned 10,000,000 credits (10,033,828)Generalized Cullen/Woodall Sieve (suspended) Jade: Earned 10,000,000 credits (10,037,204)PPS Sieve Jade: Earned 10,000,000 credits (10,305,147)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Ruby: Earned 2,000,000 credits (2,000,053)TRP Sieve (suspended) Ruby: Earned 2,000,000 credits (2,030,160)AP 26/27 Sapphire: Earned 20,000,000 credits (22,212,242)GFN Sapphire: Earned 20,000,000 credits (35,932,489)WW Emerald: Earned 50,000,000 credits (50,712,000)PSA Double Bronze: Earned 100,000,000 credits (170,761,999)
Message 149948 - Posted: 9 Apr 2021 | 16:36:43 UTC - in response to Message 149946.

This does beg the question though, does this mean other projects have sloppy programming compared to Primegrid? Could their programs be fine tuned? Or is it just that Primegrid by it's nature does more RAM access and floods the cache if HT is used? Climate prediction forums discussed a shortage of cache in a similar way. They're large data sets, so it could be the same thing as here.

Not necessarily. A proccessor has a lot of circuits designed to do different things, but it turns out the kinds of maths being done only use a few of them, so there's no room to fit in multiple instances. Other apps might use a whole bunch of stuff, and so while one circuit is busy doing one thing, you could make use of a another for a different instance that's calculating a different thing.

Obviously this is an oversimplification, but I hope it illustrates the point.

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149949 - Posted: 9 Apr 2021 | 17:29:09 UTC - in response to Message 149948.
Last modified: 9 Apr 2021 | 17:29:23 UTC

This does beg the question though, does this mean other projects have sloppy programming compared to Primegrid? Could their programs be fine tuned? Or is it just that Primegrid by it's nature does more RAM access and floods the cache if HT is used? Climate prediction forums discussed a shortage of cache in a similar way. They're large data sets, so it could be the same thing as here.

Not necessarily. A proccessor has a lot of circuits designed to do different things, but it turns out the kinds of maths being done only use a few of them, so there's no room to fit in multiple instances. Other apps might use a whole bunch of stuff, and so while one circuit is busy doing one thing, you could make use of a another for a different instance that's calculating a different thing.

Obviously this is an oversimplification, but I hope it illustrates the point.

That would mean I should be fine using some of my threads for Primegrid and some for a different type of program. Then again, would the OS know what to pair with what? Or does the CPU sort that stuff out?

Profile Rafael
Volunteer tester
Avatar
Send message
Joined: 22 Oct 14
Posts: 899
ID: 370496
Credit: 411,941,744
RAC: 209,508
321 LLR Turquoise: Earned 5,000,000 credits (8,236,942)Cullen LLR Turquoise: Earned 5,000,000 credits (8,028,695)ESP LLR Turquoise: Earned 5,000,000 credits (8,437,609)Generalized Cullen/Woodall LLR Turquoise: Earned 5,000,000 credits (8,011,297)PPS LLR Turquoise: Earned 5,000,000 credits (8,392,935)PSP LLR Turquoise: Earned 5,000,000 credits (9,998,922)SoB LLR Turquoise: Earned 5,000,000 credits (9,310,410)SR5 LLR Turquoise: Earned 5,000,000 credits (9,959,965)SGS LLR Turquoise: Earned 5,000,000 credits (8,944,665)TRP LLR Turquoise: Earned 5,000,000 credits (8,751,841)Woodall LLR Turquoise: Earned 5,000,000 credits (9,876,482)321 Sieve (suspended) Jade: Earned 10,000,000 credits (10,033,828)Generalized Cullen/Woodall Sieve (suspended) Jade: Earned 10,000,000 credits (10,037,204)PPS Sieve Jade: Earned 10,000,000 credits (10,305,147)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Ruby: Earned 2,000,000 credits (2,000,053)TRP Sieve (suspended) Ruby: Earned 2,000,000 credits (2,030,160)AP 26/27 Sapphire: Earned 20,000,000 credits (22,212,242)GFN Sapphire: Earned 20,000,000 credits (35,932,489)WW Emerald: Earned 50,000,000 credits (50,712,000)PSA Double Bronze: Earned 100,000,000 credits (170,761,999)
Message 149954 - Posted: 9 Apr 2021 | 23:26:16 UTC - in response to Message 149949.

That would mean I should be fine using some of my threads for Primegrid and some for a different type of program. Then again, would the OS know what to pair with what? Or does the CPU sort that stuff out?

That's up to the OS to figure out scheduling. As for knowing what to do with it... linux tends to do better in that regard.

Peter Hucker
Send message
Joined: 23 Sep 06
Posts: 115
ID: 3541
Credit: 46,730,776
RAC: 11
321 LLR Silver: Earned 100,000 credits (117,162)Cullen LLR Silver: Earned 100,000 credits (199,800)ESP LLR Bronze: Earned 10,000 credits (99,691)Generalized Cullen/Woodall LLR Silver: Earned 100,000 credits (100,048)PPS LLR Bronze: Earned 10,000 credits (29,800)PSP LLR Silver: Earned 100,000 credits (296,161)SoB LLR Amethyst: Earned 1,000,000 credits (1,376,215)SR5 LLR Bronze: Earned 10,000 credits (39,698)SGS LLR Bronze: Earned 10,000 credits (18,533)TRP LLR Bronze: Earned 10,000 credits (62,235)Woodall LLR Silver: Earned 100,000 credits (240,222)321 Sieve (suspended) Bronze: Earned 10,000 credits (30,838)PPS Sieve Ruby: Earned 2,000,000 credits (2,509,768)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (16,965)TRP Sieve (suspended) Bronze: Earned 10,000 credits (14,492)AP 26/27 Ruby: Earned 2,000,000 credits (4,432,423)GFN Sapphire: Earned 20,000,000 credits (36,621,124)WW Silver: Earned 100,000 credits (484,000)
Message 149969 - Posted: 10 Apr 2021 | 17:30:01 UTC - in response to Message 149954.

That would mean I should be fine using some of my threads for Primegrid and some for a different type of program. Then again, would the OS know what to pair with what? Or does the CPU sort that stuff out?

That's up to the OS to figure out scheduling. As for knowing what to do with it... linux tends to do better in that regard.

I'm sure I've read about modern CPUs doing something like this themselves, could have been something similar.

Post to thread

Message boards : Number crunching : Too much work when joining

[Return to PrimeGrid main page]
DNS Powered by DNSEXIT.COM
Copyright © 2005 - 2021 Rytis Slatkevičius (contact) and PrimeGrid community. Server load 1.10, 1.58, 1.29
Generated 18 Sep 2021 | 8:07:31 UTC