I'm pleased to see valid/invalid share stats on the dashboard. I suspect that you are not counting any shares that don't match a solution to the current block (work). If that is the case, you are doing it wrong. "stale" shares are worthless in btc, but in eth they are potentially uncle blocks. You should validate the shares against the current and previous blocks, which would capture the vast majority of the potential uncle blocks. My invalid rate was ~1% when I looked earlier today. If that is typical for the pool, then the pool is missing out on 7/8ths of that 1% (since a 1st-generation uncle gets a 7/8ths reward).
Actually I already tested similar things about uncle, and tested in reality but went out not that effective. But, now I see there's something I missed at that time. I'll implement what you stated and run it today.
I think we should not count stale shares, but try to submit new block. When stale share's block has higher difficulty than main block, then blockchain will fork and uncle candidate share would be main block and get 5 ETH.
Implemented your idea and applied to pool few minutes ago. Stale shares are treated as before (not counted unless you find the block), but pool try to submit new block if it's difficulty can make an uncle block. So, whole miner would benefit from this change in theory.
Not sure how it will make new block efficiently, because our pool is still small which is less than 2% of total. If this kind of thing were implemented in dwarf, this would have made big difference on ethereum mining world.
I just switched a rig over! I hope I got it right. I am using ethOS and all I changed was the proxypool to the ones Kotarius sighted. Do need to do anything else? I am spoiled watching the stats on ethermine . I am sending ~38-40mhs. Where should I look to see what my rig is doing? I found the Dashboard. All zero's. I will keep looking! Thanks
I've seen tcp write fails on the stratum socket in the last day or so, sometimes more than 1 per hour. The miner reconnects OK afterward. This is on us-east1. Is this related to work you are doing on the pool servers?
Not counting stale shares is fine since the number of uncle blocks produced should be proportional to the fresh shares.
Regarding the block difficulty, although I haven't looked at the code yet, from VB's writings about uncles, I don't think difficulty matters when 2 blocks at the same height are mined. The first block at a node "wins", and the 2nd block becomes the uncle, even if the ethash solution on the uncle block satisfies a higher difficulty.
Actually I read VB's writings and actual geth codes one by one when I started to test "intended uncle block mining" at first (It's about 2 months ago) And I found that geth code maintains candidate blockchains in memory pool and switch main blockchain by total difficulty. So when new block data comes, previous main block can be changed to uncle time to time.
After I implemented your idea, pool had found 3 blocks from stale shares for one day. It's quite surprising.
These blocks are all from stale shares, which means there were already higher block in network at that time. 1461038 is found to be uncle at last and included at 1461040 block. 1461079 was late block, but won total difficulty comparison and made main block to uncle block and took its place. https://etherscan.io/uncle/0x12d1966d960b091c3d8d40d749dba0f5620ba2dade8ca7d5be5edd9c2e800bcb (As looking at the blockchain data, Dwarf found it at 5/5/2016 10:30:30 AM, and our stale share found it at 5/5/2016 10:30:31 AM, but won the total difficulty comparison)
1464218 block was late block also, but became main block. Don't know which block competed with this block at that time, because their block was not included in any blockchain.
I think your idea worked quite well, and I hope this idea to be not spread to mining world I credited 5ETH Bonus to your account for precious idea. I hope you like it.
I just switched a rig over! I hope I got it right. I am using ethOS and all I changed was the proxypool to the ones Kotarius sighted. Do need to do anything else? I am spoiled watching the stats on ethermine . I am sending ~38-40mhs. Where should I look to see what my rig is doing? I found the Dashboard. All zero's. I will keep looking! Thanks
dlehenky Posts: 1,407Member ✭✭✭ May 5 Different pools have different pool difficulties. The diff on dwarfpool is 2B; the diff on ethpool is 4B. This means a "share" on dwarfpool is worth half of a "share" on ethpool. The diff on nanopool is different still, at 5B, so the "share" value there is 25% higher than ethpool. Note, however, the higher the pool difficulty, the more work the miner has to do to earn a "share".
Thanks for the explanation. Since difficulty rather than time wins, this works out even better than I thought (since some "stale" shares get the full 5 eth block reward)
The 5 eth bonus is much appreciated! MPH rules! :-)
dlehenky Posts: 1,407Member ✭✭✭ May 5 Different pools have different pool difficulties. The diff on dwarfpool is 2B; the diff on ethpool is 4B. This means a "share" on dwarfpool is worth half of a "share" on ethpool. The diff on nanopool is different still, at 5B, so the "share" value there is 25% higher than ethpool. Note, however, the higher the pool difficulty, the more work the miner has to do to earn a "share".
Mining Pool Hub uses vardiff, so every miner would get appropriate difficulty for each of them at last. New miner's difficulty starts from 2B but it can go up and down from 0.3B to 50B currently.
MiningPoolHub now treats stale shares useful, and it's time to discuss about stale share reward.
Currently, stale shares are not counted as valid share, and these are all treated as vanish work from pool. It's quite fair because rule is same for everyone. But for slow network connected miners, it's little bit not fair. They contribute more often with their stale shares but gather less than contribution.
So, we would like to change the stale share reward system. Latest top height's share (normal share) can make block and get 5 ETH. (Top height - 1) share can make uncle block and would generate 4.375 ETH. (87.5% of 5ETH) (Top height - 2) share can make uncle block and would generate 3.75 ETH. (75% of 5ETH) ....... (Top height - 6) share can make uncle block and would generate 1.25 ETH. (25% of 5ETH)
So same goes with share value too. (Top height - 1) stale share to treat as 87.5% share of normal share. (Top height - 2) stale share to treat as 75% share of normal share. ...
This rule doesn't change much about distribution rule, you would get almost same reward as before, but only pool's block finding percent stats would be higher.
I personally think it makes more sense to allow 1-block late stales at full reward. 2-blocks late, throw them out/reject. The percentage scheme seems overly complicated.
I personally think it makes more sense to allow 1-block late stales at full reward. 2-blocks late, throw them out/reject. The percentage scheme seems overly complicated.
There's two reason why I want to distinguish 1-block late shares and normal shares.
(1) late share has more probability of making uncle block than normal share. Normal share only have to overcome "the net difficulty" but late share should overcome the "net difficulty & already found block's difficulty" to be the full block.
(2) It can be abused by some bad miner. There's some vulnerability that miners would skip new work even/odd and insert one step late shares because it is more efficient than resetting every mining job. This is not tested but I expect that it would cause upto 5% difference of hashing performance.
There should be some difference between late share and normal because of this reason.
I agree with @work and @dlehenky By only accepting stale shares for the last work package (1 level back), a rogue miner intentionally mining stales would get nothing for a stale at a height of -2 instead of your proposed 75%. That should be more than sufficient to penalize rogue miners. Also a stale of height -1 is worth more than 87.5% since there is a 50/50 chance of it having a higher difficulty. So it's real weighted value would be more like 93.75%. It also makes the code on the pool side much simpler (and therefore less likely to be buggy). It also will make the dashboards stats simple since a good share of height -1 counts as a valid share. The invalid count on the dashboard would then be duplicate shares, shares of height > -1, and any other bad shares.
Lastly, this would keep mph competitive with ethermine.org since their mouthpiece has stated that the pool values stale shares the same as fresh shares. And since they apparently value stale shares of height > -1 the same as well, rogue miners would prefer ethermine over mph.
I think we've got to the point that there must be some difference between normal share and stale share. Because share's real value is mathematically different. One is better, the other is worse.
But how can we measure its value? We can't say that stale share's diff to be 50/50 chance of it having a higher difficulty. New block's difficulty value does not satisfy normal distribution of net diff.
Assume that net diff to 50, and new block's diff to 100. Then stale share's diff should be higher than 100. "Some stale share that is higher than 50 will be 50/50 chance to higher than 100" is incorrect. Maybe you think that 100 is unrealistic, but it is not incorrect even new block's diff was 60, or 70, or 80 or 90.
There's no proof that "Any number higher than 50 should be higher than number X at 50% probability". (X is higher than 50 in this sentence) Also, block difficulty spike happens all the time in real world.
Assume you get less reward after the rule change. Someone would be happy, and someone would be unhappy. Unhappy miners will leave the pool or want to argue about the share valuation percent. So to not make any complain, I tried to measure the share's mathematical expectation.
normal share's expectation value is 5 ETH. 1-late share's expectation value is 5 ETH, or 4.375ETH. Cannot measure the ratio of each case happening. (Not 50/50 chance in this case) So this is the difficult part mathematically. I once said it should be valued to 4.375 ETH (which means 12.5% drop) but as you already stated, it's quite measured lower than actual. To say 12.5% drop, normal share's expectation value should not be just 5ETH, but "5.625 ETH, or 5ETH". This ensures fair condition for each share. But ethereum spec is not like that, as you know.
This is not the end.
Uncle block's maximum inclusion number is 2. There's limit in ethereum network. There could be not enough room time to time so stale share's value must be lowered considering this condition.
Things get more complicated from 2-late share, 3-late share.. and more. They surely have some value but they would be useless when blockchain forked from parent. These late shares would be from forked chain and these shares can't make even uncle blocks.
You would think blockchain forking is abnormal. There's many many short time blocks like 2, 3, 4 seconds out there. And there's many blockchain forking. All uncles were candidate of fork. This is not abnormal behaviour, fork is always happening.
So we can't value the share perfectly in mathematically. The most realistic mathematical value seems 12.5% deduction but it's not perfect as well.
For fair distribution to satisfy all slow/fast/small/big miners, this should be considered carefully. Maybe it would be better to think about it tomorrow.
Thank you for supporting miningpoolhub. I'm not against other opinions. I just want to discuss more about it because the rule should be not changed frequently.
One more note, the reason why I want to treat all level's stale share is to remove wrong numbers for invalid share. They are actually not invalid shares but treated as invalid.
Is vardiff working properly? Share diff is moving to 0x0000000e5109ec (300 Million) and not adjusting from there. The workers page shows all my workers as inactive with 0 hashrate, but with varying difficulties. However, they ARE active, and their difficulties are all the same. Dashboard shows active hashrate. If I make a new worker and direct 150MH/s+ there, share diff rises from default to the prior mentioned target, and share rate get faster and faster.
I'm getting several shares per second, and that seems quite resource wasteful.
edit: (this is on qtminer port 20537)
Something definitely weird, because Difficulty is climbing on the workers page, but worker never gets a new share target.
Edit 2: starting a new worker causes difficulty to remain at more reasonable levels for a while, but if there is inactivity for a while, diff scales down to 300mil and doesn't seem to scale up again.
Edit 3: seem to have it working decently well at the moment. Had to delete some workers and let them be re-created.
Can you clarify what doesn't make much sense? I commented that why I have to do it in mathematical approach (to minimize complains and fair distribution), and described that there's no perfect expectation value in mathematics way.
There's nothing "wrong" with only accepting stale shares of height -1, just be transparent (i.e. document it on your pool info page).
If you accept/validate/count stale shares of height -2, -3, ... and give them different values, not only will it be more complicated for you to code on the pool side, but it will be almost impossible for pool users to make sense of.
Can you clarify why it doesn't make sense that deducting value by level? Is it just because complex? or something else?
Current pool counts (most height) shares only, and you are suggesting to count (most height, most height - 1) shares.
So I have question in opposite. (1) Why (height -1) shares should also be counted? (2) Why (height -2 , height -3 ...) shares should be not counted?
I want you to leave the reasons why you think like this. I don't understand that just stating "almost impossible for pool users to make sense of". Answering these questions would be helpful for me to understand your opinion.
I disagree that valuing the stale shares (at height -1) the same as fresh shares will lead to miners abusing the pool. And even if a couple miners did try to abuse the pool by submitting only or mostly stale shares, it should be easy for you to see in your db stats. Nevertheless, if you valued the stale shares at 85-90% of a regular share, I think most pool users would be fine with that. If you do, I think you also should update the dashboard to show valid, valid stale, and invalid shares instead of just valid & invalid.
Abusing pool attempt rate is quite high, because it's related to money. And I can't distinguish miners perfectly, because it's not that clear when miner mixes abusing strategy. What if miner attempts to abuse 1 hour period at every 3 hours mining regularly. You would think it's unrealistic, but I have seen plenty of smart people trying to get more coins from pool.
Nevertheless, if you valued the stale shares at 85-90% of a regular share, I think most pool users would be fine with that. If you do, I think you also should update the dashboard to show valid, valid stale, and invalid shares instead of just valid & invalid.
Maybe detailed stats is the most thing miner wants, not discussing stale share counting issue.
The most thing what I consider is "fair distribution". So I want to listen your opinion and reasons. If there's no appropriate. logical reasons, then I can't apply it to pool. "Fair" is the key what I consider much.
Thank you for your suggestions. Hope you answer my questions so that I can understand your thoughts.
@miningpoolhub I still think it's overly convoluted to value stale shares differently. KISS principle and all that. But, if you show it clearly to the miner, I think it could work fine. If you have multiple tiers of stale shares, then just make sure you display how many shares were submitted at each tier.
Is vardiff working properly? Share diff is moving to 0x0000000e5109ec (300 Million) and not adjusting from there. The workers page shows all my workers as inactive with 0 hashrate, but with varying difficulties. However, they ARE active, and their difficulties are all the same. Dashboard shows active hashrate. If I make a new worker and direct 150MH/s+ there, share diff rises from default to the prior mentioned target, and share rate get faster and faster.
I'm getting several shares per second, and that seems quite resource wasteful.
edit: (this is on qtminer port 20537)
Something definitely weird, because Difficulty is climbing on the workers page, but worker never gets a new share target.
Edit 2: starting a new worker causes difficulty to remain at more reasonable levels for a while, but if there is inactivity for a while, diff scales down to 300mil and doesn't seem to scale up again.
Edit 3: seem to have it working decently well at the moment. Had to delete some workers and let them be re-created.
Pool's starting diff is 2B, min diff 0.3B, max diff 50B.
2B is appropriate difficulty level for 5~6-way rig. 0.3B is for 1 GPU rig. Modern GPU get over 0.3B difficulty easily.
So when your fresh worker connects it gets 2B at first. If there's inactivity for a while, pool thinks 2B is quite high for miner. So pool decreases difficulty to 70% of current level at certain period. It can go down step by step to 0.3B finally. When miner start to insert shares, difficulty start to rise up then. But it takes time. As share insertion is probability thing, pool don't react quickly.
There's one more thing to consider. Pool remembers last diff value for each worker, so this may confuse you that even reconnection doesn't reset 0.3B diff. This is good for most miners because they would not have to deal with diff adjusting process everytime they connect.
Are you testing your custom qtminer? Above policies work well with normal miner, as far as I know.
If you don't want diff level to decrease, you have to disconnect from pool. Pool can't distinguish whether miner is idle state, or hashing but diff is too high for them.
If you want to reset miner's diff level, you have to use new workername everytime, or delete existing worker from website and reconnect. Or, you can force set starting difficulty level at connection by inserting "d={diff value}" at worker password. ex) d=2000000000 Actually I haven't tested it with ethereum miners but it should work.
@miningpoolhub I still think it's overly convoluted to value stale shares differently. KISS principle and all that. But, if you show it clearly to the miner, I think it could work fine. If you have multiple tiers of stale shares, then just make sure you display how many shares were submitted at each tier.
I agree. It's quite hard, almost impossible to value stale shares correctly. What I prefer the most is "just not counting stale shares". This is what most pool do, and have minimum complains about fair distribution.
I think stats should be improved.
Thank you about stating KISS. I almost forgot about it these days.
Yes, testing custom miner using qtminer protocol (it's the most straight-forward stratum code base to re-implement).
I'm mainly confused why a worker that goes idle for a while and climbs to 300M, but then starts spamming shares for several minutes doesn't seem to increase the share difficulty. I guess it's just quite delayed?
Edit: looks like forcing starting difficulty works as you suggested. I can use that + reconnecting as a workaround for now while I'm testing.
Re: stale shares - by not counting them, do you mean not crediting them? I personally think that at least n-1 share acceptance is a must for ethereum pools. I get a lot of n-1 stale shares on MPH, and this is one of the main reasons why I don't have all my rigs pointed here right now.
I see valuing n-1 stales at full value as quite fair. It lets work come in that's only a bit late, but reasonably prevents tampering because you can't predict if the following block will arrive in <1 second or >1 minute and cause your stale to be n-2 and therefore invalid.
Yes, testing custom miner using qtminer protocol (it's the most straight-forward stratum code base to re-implement).
I'm mainly confused why a worker that goes idle for a while and climbs to 300M, but then starts spamming shares for several minutes doesn't seem to increase the share difficulty. I guess it's just quite delayed?
Edit: looks like forcing starting difficulty works as you suggested. I can use that + reconnecting as a workaround for now while I'm testing.
Re: stale shares - by not counting them, do you mean not crediting them? I personally think that at least n-1 share acceptance is a must for ethereum pools. I get a lot of n-1 stale shares on MPH, and this is one of the main reasons why I don't have all my rigs pointed here right now.
I see valuing n-1 stales at full value as quite fair. It lets work come in that's only a bit late, but reasonably prevents tampering because you can't predict if the following block will arrive in <1 second or >1 minute and cause your stale to be n-2 and therefore invalid.
It's just delayed I think. We had some "boost" thing that climb up difficulty fast at past but it caused other problems also. It would seem quite slow, but it will get to the appropriate point at last.
About stale shares. n-1 stale share is same for other miners too. So you are not the only one getting penalized. The reward should be almost same whether the pool count stale shares or not, if you are not behind the really slow network interface. Even counting stale share will not help you to increase earnings, if you keep miners behind the slow network. The core problem is about the network, not the stale share thing.
Deducting 12.5% rule is for slow network miners who will get "some" rewards, who got nothing before. It would help "fair distribution" a little. I don't want to incentivize slow network miners to get more reward than before, I just want fair as possible. Network environment should be improved on their side.
Actually, I think modern network interface is not slow for most miners out there who can buy GPU to mine. pool <-> miner network delay is not the only thing make stales.
Why I'm constantly saying about n-2, n-3 stale shares is because it happens quite frequently. Ethereum sometimes have really short block time. 1, 2, 3, seconds of block time which causes pool's geth wallet to build up block height quickly.
So the criteria which judges n-1, n-2, n-3 is the pool's geth, and it can change very quickly. This is not related to pool <-> miner delay. pool <-> pool delay makes this happen.
It's just nature of ethereum, there's plenty of n-2, n-3 shares time to time.
Well, I like the idea of deducting 12.5% per x in n-x stale shares then. I don't think you should give credit past x=3 tho. It sounds reasonable, and wouldn't be _too_ hard to implement in the back-end and UI. It's also kinda cool, in that it follows along the Ethereum PoW reward scheme a bit.
Personally, not getting credited for stale ethash shares is a huge disadvantage to me because of very low block interval. Using a pool that doesn't count stales requires tuning my miners very differently, and the result is a large hashrate decrease. When n-1 stales are valid, I can achieve 20%+ more effective hashrate. With miners tuned for long kernel runs and running thru a local control proxy, I have peaked up to 30% rejects on MPH. I've monitored quite a bit, and I very very rarely produce any n-2 stales.
Well, I like the idea of deducting 12.5% per x in n-x stale shares then. I don't think you should give credit past x=3 tho. It sounds reasonable, and wouldn't be _too_ hard to implement in the back-end and UI. It's also kinda cool, in that it follows along the Ethereum PoW reward scheme a bit.
Personally, not getting credited for stale ethash shares is a huge disadvantage to me because of very low block interval. Using a pool that doesn't count stales requires tuning my miners very differently, and the result is a large hashrate decrease. When n-1 stales are valid, I can achieve 20%+ more effective hashrate. With miners tuned for long kernel runs and running thru a local control proxy, I have peaked up to 30% rejects on MPH. I've monitored quite a bit, and I very very rarely produce any n-2 stales.
Have you tried genoil-ethminer 1.0.8 with long kernel tuned parameters? Prior ethminer would send corrupted headerHash value by timing issue. With corrupted headerHash, pool will treat all of them as invalid share. This ethminer bug is not solved perfectly yet, as it can only handle upto n-1 shares but would solve most of the problems you are facing.
I think other miners are based from same source code also, they would have similar problem inside.
Seems like n-2 stale shares happen much when block time gets shorter. Or maybe pool delay, extreme GPU kernel settings too.
@miningpoolhub I have tried genoil's v 1.0.8, yes; glad he implemented prior headerhash lookup. I think I was one of the main people who kept pointing out that issue with ethminer.
I'm running my own private miner code, and the rejects I'm talking about aren't due to headerHash mismatch issues.
@miningpoolhub I have tried genoil's v 1.0.8, yes; glad he implemented prior headerhash lookup. I think I was one of the main people who kept pointing out that issue with ethminer.
I'm running my own private miner code, and the rejects I'm talking about aren't due to headerHash mismatch issues.
Comments
Ah.. I see.
You are really super expert in this field.
We improved pool performance greatly, and lowered latency than before.
More improvements will come soon.
Implemented your idea and applied to pool few minutes ago.
Stale shares are treated as before (not counted unless you find the block), but pool try to submit new block if it's difficulty can make an uncle block. So, whole miner would benefit from this change in theory.
Not sure how it will make new block efficiently, because our pool is still small which is less than 2% of total.
If this kind of thing were implemented in dwarf, this would have made big difference on ethereum mining world.
I hope I got it right. I am using ethOS and all I changed was the proxypool to the ones Kotarius sighted. Do need to do anything else?
I am spoiled watching the stats on ethermine .
I am sending ~38-40mhs. Where should I look to see what my rig is doing?
I found the Dashboard. All zero's.
I will keep looking!
Thanks
Seems like it is related to latest work we've done on pool. We had to restart pool service for update.
Actually I read VB's writings and actual geth codes one by one when I started to test "intended uncle block mining" at first (It's about 2 months ago)
And I found that geth code maintains candidate blockchains in memory pool and switch main blockchain by total difficulty. So when new block data comes, previous main block can be changed to uncle time to time.
After I implemented your idea, pool had found 3 blocks from stale shares for one day. It's quite surprising.
1461038 uncle with 3.75 ETH (Found from 1461040 block)
https://etherscan.io/uncle/0xa4ee1eb68f59181390d67720cf6d776d8eaf5557b7630ba890f8e58357fcafaf
1461079 - https://etherscan.io/block/1461079
1464218 - https://etherscan.io/block/1464218
These blocks are all from stale shares, which means there were already higher block in network at that time.
1461038 is found to be uncle at last and included at 1461040 block.
1461079 was late block, but won total difficulty comparison and made main block to uncle block and took its place.
https://etherscan.io/uncle/0x12d1966d960b091c3d8d40d749dba0f5620ba2dade8ca7d5be5edd9c2e800bcb
(As looking at the blockchain data, Dwarf found it at 5/5/2016 10:30:30 AM, and our stale share found it at 5/5/2016 10:30:31 AM, but won the total difficulty comparison)
1464218 block was late block also, but became main block. Don't know which block competed with this block at that time, because their block was not included in any blockchain.
I think your idea worked quite well, and I hope this idea to be not spread to mining world
I credited 5ETH Bonus to your account for precious idea. I hope you like it.
Thanks.
Have you set ethereum wallet address at wallet page?
https://ethereum.miningpoolhub.com/index.php?page=account&action=pooledit
This is mandatory work if you want to use wallet based login for mining.
If you are mining with website's username, then you don't need to set it to mine.
You can see your mining status at Dashboard but it is quite delayed info.
And here's worker list, which shows each mining rig status. It's quicker than dashboard but not accurate.
https://ethereum.miningpoolhub.com/index.php?page=account&action=workers
Paste your mining log if you can't make it work still. Seems like just wallet login problem.
Thanks.
Even the simple things evade me at times
May 5
Different pools have different pool difficulties. The diff on dwarfpool is 2B; the diff on ethpool is 4B. This means a "share" on dwarfpool is worth half of a "share" on ethpool. The diff on nanopool is different still, at 5B, so the "share" value there is 25% higher than ethpool. Note, however, the higher the pool difficulty, the more work the miner has to do to earn a "share".
You're welcome!
Have a nice day
Thank you for precious info.
Mining Pool Hub uses vardiff, so every miner would get appropriate difficulty for each of them at last.
New miner's difficulty starts from 2B but it can go up and down from 0.3B to 50B currently.
Small and big miners are all welcomed.
MiningPoolHub now treats stale shares useful, and it's time to discuss about stale share reward.
Currently, stale shares are not counted as valid share, and these are all treated as vanish work from pool.
It's quite fair because rule is same for everyone. But for slow network connected miners, it's little bit not fair. They contribute more often with their stale shares but gather less than contribution.
So, we would like to change the stale share reward system.
Latest top height's share (normal share) can make block and get 5 ETH.
(Top height - 1) share can make uncle block and would generate 4.375 ETH. (87.5% of 5ETH)
(Top height - 2) share can make uncle block and would generate 3.75 ETH. (75% of 5ETH)
.......
(Top height - 6) share can make uncle block and would generate 1.25 ETH. (25% of 5ETH)
So same goes with share value too.
(Top height - 1) stale share to treat as 87.5% share of normal share.
(Top height - 2) stale share to treat as 75% share of normal share.
...
This rule doesn't change much about distribution rule, you would get almost same reward as before, but only pool's block finding percent stats would be higher.
We want to listen about your opinions.
Whatever rule is considered, miners would get more reward with latest genoil-ethminer 1.0.8
https://github.com/Genoil/cpp-ethereum/tree/108/releases
There's two reason why I want to distinguish 1-block late shares and normal shares.
(1) late share has more probability of making uncle block than normal share.
Normal share only have to overcome "the net difficulty"
but late share should overcome the "net difficulty & already found block's difficulty"
to be the full block.
(2) It can be abused by some bad miner.
There's some vulnerability that miners would skip new work even/odd
and insert one step late shares because it is more efficient than resetting
every mining job.
This is not tested but I expect that it would cause upto 5% difference of hashing performance.
There should be some difference between late share and normal because of this reason.
You mean that hashrate does not drop even job reset?
I think we've got to the point that there must be some difference between normal share and stale share. Because share's real value is mathematically different. One is better, the other is worse.
But how can we measure its value?
We can't say that stale share's diff to be 50/50 chance of it having a higher difficulty.
New block's difficulty value does not satisfy normal distribution of net diff.
Assume that net diff to 50, and new block's diff to 100.
Then stale share's diff should be higher than 100. "Some stale share that is higher than 50 will be 50/50 chance to higher than 100" is incorrect.
Maybe you think that 100 is unrealistic, but it is not incorrect even new block's diff was 60, or 70, or 80 or 90.
There's no proof that "Any number higher than 50 should be higher than number X at 50% probability". (X is higher than 50 in this sentence)
Also, block difficulty spike happens all the time in real world.
Assume you get less reward after the rule change.
Someone would be happy, and someone would be unhappy. Unhappy miners will leave the pool or want to argue about the share valuation percent.
So to not make any complain, I tried to measure the share's mathematical expectation.
normal share's expectation value is 5 ETH.
1-late share's expectation value is 5 ETH, or 4.375ETH. Cannot measure the ratio of each case happening. (Not 50/50 chance in this case) So this is the difficult part mathematically.
I once said it should be valued to 4.375 ETH (which means 12.5% drop) but as you already stated, it's quite measured lower than actual. To say 12.5% drop, normal share's expectation value should not be just 5ETH, but "5.625 ETH, or 5ETH". This ensures fair condition for each share. But ethereum spec is not like that, as you know.
This is not the end.
Uncle block's maximum inclusion number is 2. There's limit in ethereum network. There could be not enough room time to time so stale share's value must be lowered considering this condition.
Things get more complicated from 2-late share, 3-late share.. and more. They surely have some value but they would be useless when blockchain forked from parent. These late shares would be from forked chain and these shares can't make even uncle blocks.
You would think blockchain forking is abnormal. There's many many short time blocks like 2, 3, 4 seconds out there. And there's many blockchain forking. All uncles were candidate of fork. This is not abnormal behaviour, fork is always happening.
See below links and compare.
https://etherscan.io/charts/uncles
https://etherscan.io/charts/blocktime
There's no guarantee for uncle making ratio/count at all at ethereum network.
So we can't value the share perfectly in mathematically. The most realistic mathematical value seems 12.5% deduction but it's not perfect as well.
For fair distribution to satisfy all slow/fast/small/big miners, this should be considered carefully. Maybe it would be better to think about it tomorrow.
Thank you for supporting miningpoolhub. I'm not against other opinions. I just want to discuss more about it because the rule should be not changed frequently.
I'm getting several shares per second, and that seems quite resource wasteful.
edit: (this is on qtminer port 20537)
Something definitely weird, because Difficulty is climbing on the workers page, but worker never gets a new share target.
Edit 2: starting a new worker causes difficulty to remain at more reasonable levels for a while, but if there is inactivity for a while, diff scales down to 300mil and doesn't seem to scale up again.
Edit 3: seem to have it working decently well at the moment. Had to delete some workers and let them be re-created.
I commented that why I have to do it in mathematical approach (to minimize complains and fair distribution), and described that there's no perfect expectation value in mathematics way.
Can you clarify why it doesn't make sense that deducting value by level?
Is it just because complex? or something else?
Current pool counts (most height) shares only, and you are suggesting to count (most height, most height - 1) shares.
So I have question in opposite.
(1) Why (height -1) shares should also be counted?
(2) Why (height -2 , height -3 ...) shares should be not counted?
I want you to leave the reasons why you think like this. I don't understand that just stating "almost impossible for pool users to make sense of". Answering these questions would be helpful for me to understand your opinion.
Actually there's other community forum I asked also.
http://www.ddengle.com/pool_mining_pool_hub/1229176
They prefer "not count stale shares at all" or "count all stale shares deducting it's value 12.5%".
Nobody wanted counting only (height -1) actually.
Abusing pool attempt rate is quite high, because it's related to money. And I can't distinguish miners perfectly, because it's not that clear when miner mixes abusing strategy. What if miner attempts to abuse 1 hour period at every 3 hours mining regularly. You would think it's unrealistic, but I have seen plenty of smart people trying to get more coins from pool.
Maybe detailed stats is the most thing miner wants, not discussing stale share counting issue.
The most thing what I consider is "fair distribution". So I want to listen your opinion and reasons. If there's no appropriate. logical reasons, then I can't apply it to pool. "Fair" is the key what I consider much.
Thank you for your suggestions. Hope you answer my questions so that I can understand your thoughts.
Pool's starting diff is 2B, min diff 0.3B, max diff 50B.
2B is appropriate difficulty level for 5~6-way rig.
0.3B is for 1 GPU rig. Modern GPU get over 0.3B difficulty easily.
So when your fresh worker connects it gets 2B at first. If there's inactivity for a while, pool thinks 2B is quite high for miner. So pool decreases difficulty to 70% of current level at certain period. It can go down step by step to 0.3B finally.
When miner start to insert shares, difficulty start to rise up then. But it takes time. As share insertion is probability thing, pool don't react quickly.
There's one more thing to consider. Pool remembers last diff value for each worker, so this may confuse you that even reconnection doesn't reset 0.3B diff. This is good for most miners because they would not have to deal with diff adjusting process everytime they connect.
Are you testing your custom qtminer? Above policies work well with normal miner, as far as I know.
If you don't want diff level to decrease, you have to disconnect from pool. Pool can't distinguish whether miner is idle state, or hashing but diff is too high for them.
If you want to reset miner's diff level, you have to use new workername everytime, or delete existing worker from website and reconnect.
Or, you can force set starting difficulty level at connection by inserting "d={diff value}" at worker password. ex) d=2000000000
Actually I haven't tested it with ethereum miners but it should work.
I agree. It's quite hard, almost impossible to value stale shares correctly.
What I prefer the most is "just not counting stale shares". This is what most pool do, and have minimum complains about fair distribution.
I think stats should be improved.
Thank you about stating KISS. I almost forgot about it these days.
Yes, testing custom miner using qtminer protocol (it's the most straight-forward stratum code base to re-implement).
I'm mainly confused why a worker that goes idle for a while and climbs to 300M, but then starts spamming shares for several minutes doesn't seem to increase the share difficulty. I guess it's just quite delayed?
Edit: looks like forcing starting difficulty works as you suggested. I can use that + reconnecting as a workaround for now while I'm testing.
Re: stale shares - by not counting them, do you mean not crediting them? I personally think that at least n-1 share acceptance is a must for ethereum pools. I get a lot of n-1 stale shares on MPH, and this is one of the main reasons why I don't have all my rigs pointed here right now.
I see valuing n-1 stales at full value as quite fair. It lets work come in that's only a bit late, but reasonably prevents tampering because you can't predict if the following block will arrive in <1 second or >1 minute and cause your stale to be n-2 and therefore invalid.
It's just delayed I think. We had some "boost" thing that climb up difficulty fast at past but it caused other problems also. It would seem quite slow, but it will get to the appropriate point at last.
About stale shares.
n-1 stale share is same for other miners too. So you are not the only one getting penalized. The reward should be almost same whether the pool count stale shares or not, if you are not behind the really slow network interface.
Even counting stale share will not help you to increase earnings, if you keep miners behind the slow network. The core problem is about the network, not the stale share thing.
Deducting 12.5% rule is for slow network miners who will get "some" rewards, who got nothing before. It would help "fair distribution" a little.
I don't want to incentivize slow network miners to get more reward than before, I just want fair as possible. Network environment should be improved on their side.
Actually, I think modern network interface is not slow for most miners out there who can buy GPU to mine.
pool <-> miner network delay is not the only thing make stales.
Why I'm constantly saying about n-2, n-3 stale shares is because it happens quite frequently.
Ethereum sometimes have really short block time. 1, 2, 3, seconds of block time which causes pool's geth wallet to build up block height quickly.
So the criteria which judges n-1, n-2, n-3 is the pool's geth, and it can change very quickly. This is not related to pool <-> miner delay. pool <-> pool delay makes this happen.
It's just nature of ethereum, there's plenty of n-2, n-3 shares time to time.
Well, I like the idea of deducting 12.5% per x in n-x stale shares then. I don't think you should give credit past x=3 tho. It sounds reasonable, and wouldn't be _too_ hard to implement in the back-end and UI. It's also kinda cool, in that it follows along the Ethereum PoW reward scheme a bit.
Personally, not getting credited for stale ethash shares is a huge disadvantage to me because of very low block interval. Using a pool that doesn't count stales requires tuning my miners very differently, and the result is a large hashrate decrease. When n-1 stales are valid, I can achieve 20%+ more effective hashrate. With miners tuned for long kernel runs and running thru a local control proxy, I have peaked up to 30% rejects on MPH. I've monitored quite a bit, and I very very rarely produce any n-2 stales.
Have you tried genoil-ethminer 1.0.8 with long kernel tuned parameters? Prior ethminer would send corrupted headerHash value by timing issue. With corrupted headerHash, pool will treat all of them as invalid share. This ethminer bug is not solved perfectly yet, as it can only handle upto n-1 shares but would solve most of the problems you are facing.
I think other miners are based from same source code also, they would have similar problem inside.
Seems like n-2 stale shares happen much when block time gets shorter. Or maybe pool delay, extreme GPU kernel settings too.
I'm running my own private miner code, and the rejects I'm talking about aren't due to headerHash mismatch issues.
You aren't accepting n-1 shares yet, right?
Not yet. Calculates all of them and build block if it can but not counted. All of them are invalid currently.