Blockchain at 94% of a 50GB drive (and I'm not yet current)!

motoaddictmotoaddict Fort Myers, FloridaMember Posts: 52
This seems unreasonable for a test blockchain and unsustainable in the long run. What am I missing?

Comments

  • robmyersrobmyers Member Posts: 65 ✭✭✭
    Which version of the client are you running?
  • motoaddictmotoaddict Fort Myers, FloridaMember Posts: 52
    Currently at "instance: Geth/v0.9.36/linux/go1.4.2".
  • o0ragman0oo0ragman0o Member, Moderator Posts: 1,291 mod
    I've questioned this too. But it doesn't seem to be the blockchain which is only 2.2GB but the 'state' instead which is over 53Gb.

    I don't understand the State and need to reread the yellow paper or something.
  • o0ragman0oo0ragman0o Member, Moderator Posts: 1,291 mod
    0.9.36 is said to have addressed the size of state.
  • jzenjzen Member Posts: 49
    Yes, but to maximally benefit, the client's local blockchain/state database needs to be reset.
  • motoaddictmotoaddict Fort Myers, FloridaMember Posts: 52
    "Reset"? You mean wipe the db and start the download anew??
  • motoaddictmotoaddict Fort Myers, FloridaMember Posts: 52
    I'm guessing "yes". Here we go again!
  • jzenjzen Member Posts: 49
    Yes. But of course, there is a down-side to resetting at this point, it'd mean having your client attempt to sync starting from block zero, and that'd be a long climb back up though it'd be an interesting test. Since running 0.9.36 the size of my .ethereum/state directory has not grown significantly as it had been doing while running previous versions. If you can live with the space it's already occupying, leave it be.
  • jzenjzen Member Posts: 49
    @motoaddict Good luck! I'd be curious to know the results of your test, how long it takes to sync and the size of the state when current.
  • motoaddictmotoaddict Fort Myers, FloridaMember Posts: 52
    Back up to about 618K blocks this morning and new blocks slowing to a crawl. Seems a little ridiculous to have a test blockchain this large. Can't mine, receive ether, test contracts, etc. without an up to date blockchain (at least that's my understanding). Really an impediment to new users. I've never had a complete ethereum blockchain. :(
  • DonaldMcIntyreDonaldMcIntyre DenverMember Posts: 17
    I hope the normal blockchain, when it launches, is downloadable by anybody. This test-net is not suitable for mass markets and thus a restriction in Ethereum adoption.

    If everybody needs to use Ethereum through hosted services then we are talking about an enterprise back-end service, not a public "free" network.

    Also, we are talking about re-centralization, since only through big corporate nodes it is possible to use Ethereum.

    Am I being too catastrophic?
  • SmithgiftSmithgift Member Posts: 64
    There were/are prizes for spamming the current blockchain (with free test ether), so that it could be stress tested. This hopefully will not happen to the real network.
  • tym4ttym4t Nagata-ku, Kobe-shi, Hyogo-ken, JapanMember Posts: 71 ✭✭✭
    If you have a version >= 0.9.36 it is recommended that you clear your state and block directories then sync from the beginning. If you want to do it faster then try out @StephanTual 's MKII experiment

    It's unlikely that we will see the type of TX volume we did on Olympic since we won't have many people who will just throw away their Ether to spam Frontier.

    Ultimately state pruning needs to be addressed as this is going to pose a major usability problem for most people. It would be hard to convince people to download Mist to install DApps when users have to wait 2 days just to be able to use them. Most people are already impatient with sites that have more than a 5 second page load time.

    Maybe PoS based hypercubes or letting sub-chains decide on the trustworthiness of other sub-chains seem to be the most reasonable solution. I personally like the idea of having separate chain for each DApp that checkpoint with other various chains of their liking similar to the concept of CPU affinity where threads can pin themselves to one or more CPU cores or thread manager. Each DApp can choose which sub-chains they will look towards for consensus to be determined either from creation or dynamically set by certain rules/stakeholder voting.
  • oliverkxoliverkx Member Posts: 85
    I agree that requiring every mining node to process every transaction for every contract is a scalability nightmare. Bitcoin got away with a single chain due to the fact it mostly just processes transfers from one account to another: the processing power and amount of data involved are relatively trivial. Ethereum on the other hand wants to do far more complex operations, requiring more data and more CPU power, and all this in 12 second block times. If millions of people, not to mention IoT appliances adopt the platform, we may be looking at millions of transactions per block. And a terabyte block-chain.

    I suppose that 100 miners can handle the workload for 100,000 users (1 miner per 1,000 users).
    But could 100,000 miners handle the workload for 100,000,000 users (same miner per 1,000 users ratio)?

    Not sure what the solution is, but these are fundamental questions that I feel aren't being adequately addressed by the core group. I see a lot of meetings, Devcons, and lot's of exciting ideas. But ultimately, none of these ideas really matter if most people aren't able to download the block-chain.

    I have very high hopes for Ethereum. But months have passed, and these basic issues in the client code have neither been resolved, nor have they even been openly discussed by the core Devs (that I am aware of).

    My feeling is that the top 2 issues that MUST be addresses before anything else are:

    1 - Bring performance when downloading the block-chain to an acceptable level
    (stipulate a target download hours per block-chain history years)

    2 - Stability when mining
    (make sure that mining nodes don't fall behind and mine stale blocks)

    NOTE: to be honest, #2 might have been addressed already, and I wouldn't know because #1 has kept me from mining for a few weeks now.

    As far as #1 goes, if the goal can be achieved by fixing bugs in the clients, great! But if it cannot, then now is probably the time to rethink some of the fundamentals, such as sub-chains, or other ways to achieve scalability.

    The alternative would be that some third-party is going to use the principles and ideas behind Ethereum, as well as bits and pieces of its open-source code, and build a competing solution that addresses the problems that Ethereum didn't address quickly enough on its own. And while in the long run, this might not make a big difference to humanity as a whole, I still would prefer the current project (and the Ethers I have invested in it) to carry the day!
  • jzenjzen Member Posts: 49
    @oliverkx Really good points. Regarding #2, the issue has not yet been resolved, based on my recent experience, a mining node running current geth version 0.9.39 stumbles on occasion, falls behind, and then mines stale blocks until geth process is restarted.
  • oliverkxoliverkx Member Posts: 85
    Thanks jzen! Downloading around block #790K. Will see when we get there...
  • mikmik Member Posts: 57 ✭✭
    Struggling to catch up with the blockchain for more than 2 weeks, I could finally catch up again with the new release (0.9.39) within a few (~10) hours :) AFTER deleting the old bc!
  • oliverkxoliverkx Member Posts: 85
    I also restarted from scratch with build 0.9.39 - now around block #813K, about 40+ hours into it.
  • oliverkxoliverkx Member Posts: 85
    Almost caught up this morning, but the client got stuck at block #869370 which (by my calculation) must have been about the last block in the chain at the time. Geth won't even shut down at this point, so I am going to have to force it down and restart...
  • mikmik Member Posts: 57 ✭✭
    I suggest to backup .ethereum whenever you shut down geth. That's wat I'm doing now and what I'm continuing to do ;)
Sign In or Register to comment.