Alpereum ETH Pool back online, after global 'Geth' client issue

BitcoinSuisseAGBitcoinSuisseAG Baar, Zug, SwitzerlandMember Posts: 37
Today at around 20:00 CET (Paris time) Sep. 18th 2016, the 'Geth' clients on our pool started failing / producing corrupt blockchain data, forcing our pool services to a halt.

It was quickly determined that this was caused by a general problem with the 'Geth' client and the network, and Alpereum was quickly switched to global use of the 'Parity' client, but a speedy relaunch was prevented by the need for the Parity clients on our servers to fully syncronize.
Several descriptions and discussions on the geth problem are available at: https://www.reddit.com/r/ethereum

As of 04:25 CET, Sept 19th, the Alpereum 'Parity' nodes had syncronized, and the pool relaunched on the 'Parity' client, with full functionality, except payouts, which are disabled pending thorough tests on 'Parity', but balances will be updated and kept.

Payouts are expected to be functioning as well within the next 12 hours, and no funds will be lost in the meantime.

For the loss of service in the 8 hours Alpereum services were unavailable, and altough strictly not an Alpereum issue, all our clients with were with us prior to the Geth issue, are subject to a refund from us for the downtime suffered.
Send us a worker link at [email protected], along with estimated ETH loss, and we'll verify and pay the loss-of-service refund directly to the mining wallets.

On Aplereum.ch we'll keep uptating on the issue, until all test unanimously point to the issue being fully resoleved.

Thank you for your patience and understanding. We will continue to monitor the situation carefully.
www.alpereum.ch

Comments

  • BitcoinSuisseAGBitcoinSuisseAG Baar, Zug, SwitzerlandMember Posts: 37
    Balances (back to 04:00am CET this morning and to present) are now correctly updated, and blocks found are now being correctly accounted for.

    Individual balances, even if above 0.2 ETH, are still not being paid out, pending testing being completed on our Parity payout module, but also Payouts will soon be engaged, restoring full functionality to the Pool.

    Manual reimbursement for (generously set) 10h of downtime for all workers, who were mining against Alpereum, when the Geth-issues started 19:58 CET on the 18th, and all the way to 04:00 CET on the 19th, will commense in 4 hours.

    We are working hard on solving the issues in the best possible fashion, and not a Weii will be lost for any Alpereum client, from downtime or otherwise.

    Thank you for your patience!
  • BitcoinSuisseAGBitcoinSuisseAG Baar, Zug, SwitzerlandMember Posts: 37
    Alpereum Status Update - 'Geth Issue'
    Update: September 21st, 02:00 CET

    After the Geth-issue on Septh 18th, which so dramatically affected the Ethereum network. the Alpereum Ether Pool migrated to the Parity daemon (1.3.0, https://get.parity.io .deb) in the morning hours of Sept 19th.

    Due to the Geth issue, the pool suffered complete failure between Sept 18th 19:58 CET and Sept 19th 04:00 CET, with no data being recorded, a total of 8 hours.
    The complete Network Hashrate at the time was 5.4 TH/s.

    Alpereum chooses to compensate its miners for a full 10 hours of non-stop mining, the additional 2 hours compensating stability issues which also plagued the pool outside the downtime.
    Miner compensations are based on the average hashrate before the crash, including stale shares over 10 hours, and multiplied with 1.25 (25 % added for good measure).
    These compensations were out manually, to all miners on September 20th/21th 2016, according to this table.

    After the migration to Parity, it was found that the Parity client did not perform to expectations. Synchronization issues kept plaguing the servers, and orphaned blocks were found, out of proportion to what would be expected.
    As the Geth 1.4.12 patch in the meantime had been released, we chose to abandon Parity and revert back to the tried-and-tested, now updated Geth 1.4.12, which had been patched to address the original Geth-issue.

    Thus from Sept 19th 04:00 CET and all the way to when the pool was restored to Geth 1.4.12 (Sept 20th, 22:30 CET) the external miners on our pool in the period, also have suffered what would appear to be sub-performance.
    We gathered the data to compensate them as well, for the full 42.5 hours we ran with Parity, in a similar fashion as to the downtime compensations in the table, listed above.
    These compensations were out manually, to all miners on September 20th/21th 2016, according to this table.

    All functionality, including payouts on the Alpereum Pool, has been fully restored, and the pool has been running normally on Geth 1.4.12 since 23:30 CET, September 20th 2016.
    There were no downtime in connection with daemon switch from Parity back to Geth, and testing with Parity will continue moving forwards. Alpereum can now seamlessly switch between the Geth and Parity, and indeed, run both at the same time.
    Experimentation with Parity will continue, but not on the main servers, before stability and performance can be guaranteed.

    In short, even when the whole Ethereum network is on fire, not a Weii has been lost for those, mining against the Alpereum ETH Pool. Something to consider when selecting your pool in the future.

    This concludes the Geth-issue Sept 18th-20th 2016, and we feel confident all our miners will be compensated appropriately for the downtime.
    We point out that this serious issue was due to no fault on the part of Alpereum, but nevertheless take our commitment to service excellence seriously.

    Happy mining, Everyone!

    The Alpereum team
    [email protected]
Sign In or Register to comment.