What would a 'trusted data feed' look like?



  • agamemnonagamemnon Member Posts: 7
    > The data feed source needs to release a signed data feed, and then the contracts will
    > verify the signature with the data feed's public key in script code.

    how does the source and contract communicate?
  • edmundedgaredmundedgar Member Posts: 19 ✭✭
    > Q: how do you pull in external data? (eg. weather, EURUSD cross, etc.)
    > A: The data feed source needs to release a signed data feed, and then the contracts will verify the signature with the data feed's public key in script code.

    If anyone wants to give me a specification for what Ethereum wants these feeds and signatures to look like we can go ahead and start publishing them...
  • cyberfaxcyberfax Member Posts: 11 ✭✭
    Distributed sensors networks could be quite interesting. Each sensor could have a private key to sign the data it will send to an Ethereum contract, and as an incentive to add more sensors and to publish correct data, the sensor could get paid for the data it submits. The more trusted some data is, the higher amount it would get paid. Initially the payment could be very low, but as it increases its trust, the pay would increase.

    The data from each sensor could also be verified by the network, to give it some kind of trust rank.

    So anyone could earn a bit of money, by for example placing temperature sensors around their property. If the data they supplied were deemed valuable and correct, they'd get paid.

    There could be a web of trust built so that people or agents could vote on trustworthiness of different sensors, and their trust would then reflect how much you could trust the data. You might trust someone, who trusts someone else, who trusts that a certain sensor provides reasonably correct data.

  • chris613chris613 Member Posts: 93 ✭✭
    Cyberfax has some cool ideas. agamemnon asked how the two communicate, to which I think the answer is the EXTRO opcode, which is block.contract_storage() in CLL I think. This lets you read the storage items of another contract. So AFAICT a feed would publish data into it's own storage area for reading by other contracts. I could imagine two contracts doing payment/key negotiation for a key to access a paid datafeed, but the storage and runtime requirements seem pretty large so we'll have to see how the economics of these things play out
  • sl0anagesl0anage Member Posts: 13
    Hey guys, has this conversation branched off elsewhere? Anyone still talking about this somewhere I can get involved? Thanks
  • mids106mids106 Member Posts: 188 ✭✭✭
    EXTRO is removed from POC3.5, now you need to CALL a datafeed contract and get a RETURN with the requested value; this allows datafeeds to charge for their services! Yay!
  • JasperJasper Eindhoven, the NetherlandsMember Posts: 514 ✭✭✭
    @agamemnonagamemnon: source and contract communicate with transactions, transactions hold data.(has actually been said in this thread and whitepaper)

    @mids106: it might still be useful. But anyone can read the state of the datasharing contract from the blockchain. You could just use the transaction to provide that data. The developer would have to actually figure out how to read it, but it is hidden by obscurity at best, and the developer could just follow the logic of the contract in a case where it accepts to return the data.

    Starting to wonder more: what does and doesnt need distributed consensus? I suppose we might find out when the script running costs go up and there is incentive to get stuff off the blockchain.

    Does this data thing need it? It needs trust in data sources, signed reports of what data is, and a decentralized network. That is able to catch people if they make conflicting claims.

    But it does if you want to monetize it, or provably collect the data in a particular way.
  • chris613chris613 Member Posts: 93 ✭✭
    @mids106? - The way the CALL/RETURN model allows a datafeed to charge for their services is very interesting to me. On the surface, of course, this is absurd; the storage and contract logic are public, so the data is literally free to anyone with a copy of the blockchain. But you have to get it into other contracts by either feeding it in with a transaction as Jasper suggested, or by setting up your own "pirate" version of the feed that will give it for free, maybe just to your contract if you want. The cost to do this is a minimal investment in IT required to maintain the mirror, whereas the cost to actually produce the original feed is usually quite a bit higher. Very hostile market conditions, as music and movie producers will attest.

    The interesting twist on this is that people transacting with your contract can see which feed your are using, or if you are just providing your own data. Presumably if they wanted the outcome of a transaction to depend on some data from a feed, they would want you to actually use that feed and pay the fee, otherwise they are trusting you instead of the factual authority. Therefore there appears to be intrinsic value to authoritative data, and I can certainly see companies setting up legal contracts that require blockchain contracts to consume certain feeds in order to fulfil the requirements of the agreement. In fact I think that finding partners willing to incorporate your feed into such agreements is one of the only real paths to monetization of a feed. It's not clear how the current body of IP law would handle contract storage where there is no clear way to indicate that access requires a license, and circumvention is so trivial even by DRM standards. IANAL but I doubt "you need to look it up in the blockchain where it very publicly stored it" could be considered a "digital lock". Uncertainty around this issue is likely to be a barrier to commercialization of the simpler fee-for-datafeed models except at very small scale.
  • SatoshiFantasySatoshiFantasy Member Posts: 3
    edmundedgar you may be interested in satoshifantasy. white paper and alpha code release date 4/15
  • sl0anagesl0anage Member Posts: 13
    Okay, I'm still catching up on all the terminology used here. From what I gather there is now a way to get a datafeed into the blockchain through some CALL/RETURN model I'm unfamiliar with (would like a reference to solid explanation if possible) and these sources can choose to charge for their data inclusion.

    If there was a way to get data about nominal GDP or some other macroeconomic variable as part of this datafeed, could the production algorithm in a blockchain find a way to include this variable in an distributed-consensus way?

    Essentially, I want to find a way to gather a globally accepted input into the blockchain that changes the output of coin according to nGDP (or total sum of spending in the economy).

    Thoughts? Feel free to tell me to stfu too...still in the research phase of this idea.

    ^ talks about including velocity to control output, but what about nGDP?
  • chris613chris613 Member Posts: 93 ✭✭
    @sl0anage? , the basic explanation of CALL/RETURN is that contract code can "call" another contract which can then, in turn, "return" a value back. The called contract could only accept calls that send along a certain amount of ether, and thereby charge for the data (subject to the issues I described in my last post).

    Someone could quite easily publish a data feed of the nGDP of a given country and make it available for other contracts to act upon using a call to the data feed contract. There is no built-in distributed-consensus algorithm for the validity of data feed data itself, but if you could find N data feeds publishing the nGDP data you want, you could implement your own M of N agreement algorithm to verify the data in a consensus fashion.
  • sl0anagesl0anage Member Posts: 13
    @chris613? what would this data feed contract look like?
  • chris613chris613 Member Posts: 93 ✭✭
    @sl0anage?, the most trivial example (completely untested, too!) would look something like this:

    FEED_FEE =

    if (tx.sender == OWNER_ADDR):
    contract.storage[block.timestamp] = tx.data[0]

    if tx.value > FEED_FEE:
    return contract.storage[tx.data[0]]

    This allows the owner to send in a data element that gets stored at an address corresponding to the last block's timestamp, so you can only store one value per block. In your example, when a block gets published on the network, you would feed in the GDP as of that time. Consumers call this contract and pass in the block timestamp corresponding to the timeperiod they are interested in; if the fee is sufficient the contract returns the data value.

    Some trivial refinements come to mind, but I wanted to show you the simplest case so it's very clear. Here are some major improvements you might want:
    * Feed in a timestamp along with the data point and return closest match to any requested timestamp
    * Allow requesting of statistical functions (average, max, min, etc) over a range of data points
    * Have a registration process so that consumers can receive data immediately when it is fed in

    In the beginning of this thread we talked about publishing signed data. At first I took this to mean publishing actual signature data into the feed contract's storage for other contracts to verify via crypto functions. After further study and working with the contract language I now realize that the transaction where OWNER_ADDR sends the data element into the feed is actually the signed version of that data that we were looking for. The consumer contract doesn't need to do signature verification at all, it was already verified by the transaction validation code. Since it's always public knowledge where a consumer contract is getting its data, you can trust that the consumer contract is only operating on legitimate, signed data from that feed.

    Corrections welcome if anyone thinks I've mis-understood or mis-explained.
  • chris613chris613 Member Posts: 93 ✭✭
    I had angle-bracketed versions of "your address" and "required fee" on the first two lines, but the forum ate them
  • sl0anagesl0anage Member Posts: 13
    edited March 2014
    @chris613? please allow me to chew through your pseudo here. Also, I'm typing in a free-flow style so as my confusion becomes eminent, I truly apologize.

    We have an "owner address" aka "your address" that is associated with who's address, the owner of the datafeed? If, for instance, there's a data feed from the federal reserve where ethereum can access the data from the fed's API or something, would the fed's feed be the owner address?

    the "tx.data[0]" would be equivalent to this nGDP feed, right?

    The proposal I'm making would be to create a coin that uses the datafeed from this federal reserve API to control coin output based on the nominal GDP datafeed. From what I can see, your code is to be used and called from other contracts to include arbitrarily and if they pay the fee to the "owner" of the data.

    Granted, this seems to be something that belongs in the "other cryptocurrency" forum topic but datafeeds are essential to this proposal, and demands addressing first!
  • chris613chris613 Member Posts: 93 ✭✭
    Yes, in your example the federal reserve is the owner of the feed and tx.data[0] is the data element that was fed in with the transaction that activated the contract - in this case the nGDP as of the timestamp of the last block.

    You could definitely control the supply of a sub-currency using this sort of feed. Here is a trivial sub-currency example: https://github.com/WeMeetAgain/ethereum-example-contracts/blob/master/sub-currencies/subcurrency_whitepaper.cll

    This contract gives 10^18 coins to the creator and then he would hand them out from there. In your contract you'd need logic to track the current supply, call the nGDP feed, and then distribute/destroy holdings according to your algorithm.

    PS. That wasn't pseudo code. Except for the initialization of the two variables and some spacing that got eaten by the forum software, that is actual (though untested) ethereum contract code in CLL.
  • sl0anagesl0anage Member Posts: 13
    What do you think the benefits of building a new coin with ethereum contracts versus an competing altcoin with these properties?

    Also, can you point me to an explanation of exactly how the ethereum system plans to leverage bitcoin? I'm struggling to grasp how the ethereum data will be included and how, with so many potential apps from ethereum code, the data can be manageable on the blockchain?
  • chris613chris613 Member Posts: 93 ✭✭
    @sl0anage?, I don't really have an opinion on the contract vs. altcoin. I think the point of ethereum is that it is a platform where building the type of thing you are describing is fairly trivial compared to worrying about the entire crypto underpinnings of the system. Might as well use it. Also it's much more likely you'll get parties willing to publish feeds onto a system where it will get a wider exposure, so if ethereum gains adoption that's another reason to use it as well.

    As for bitcoins, I don't think they are planned to be used within ethereum. It sounds like they might be accepted as fundraising for the initial investment in the platform, but beyond that there is no strict coupling built in.

    Finally, the question of whether the data will be manageable if everything is built on one blockchain. Many suspect the answer is 'no', so techniques for provably bridging blockchains are being worked on. There have been a few discussions that touch on this, but I couldn't find anything too concrete.
  • JasperJasper Eindhoven, the NetherlandsMember Posts: 514 ✭✭✭
    @chris613 but they could also have choosen to make it trivial to start new blockchains. The thing is that it would proliferate different values for different currencies, and contracts cant talk across blockchains.

    Non-contracts(humans) can talk across blockchains, btw. There 'S releases a transaction using H(S)' method. Been writing that. If clients could do the communication well, it could be transparent.

    I wonder if a 'simply trust lightweight clients' would work. Basically most people would be full node on a couple of merged-mining blockchains and would verify whatever not on those blockchains by using the lightweight clients of those blockchains.

    I went off topic. The SchellingCoin blog post is relevant to this discussion. I am critical about the idea that 'the truth is the best Schelling point'. I mean the value of bitcoins and/or ethers itself is a Schelling point. It essentially isnt a truth.

    However the point isnt for the value to be actual value to be 'the truth', the point is that the value changes smoothly.
  • Olga_MOlga_M Member Posts: 1
    Can this be a DAC? Basically, an original set of members can start posting a data feed, with the verified answer requiring some percentage of the votes to agree. Or else a limited member prediction market, where we would expect most predictions to be the same. Others could pay a transaction cost to peg their contract to the data feed, which would be used to pay the predictors and incentivize accuracy. Any new members would have to be invited by a majority vote, reducing the chances of mass collusion.

    Someone should tell me if this would work.
  • edmundedgaredmundedgar Member Posts: 19 ✭✭
    edited May 2015
    OK, so we're just preparing to implement Ethereum-native data feeds, and thought I'd publish the way we're planning on doing it to see if anyone has any comments or suggestions. We haven't done much work on Ethereum so this may result in some forehead-slapping. Please don't be shy about telling us.

    To recap, our current process is:
    1) You hit our website to register something you're interested in on our site as a binary proposition (1 USD to be > 100 JPY on June 1st). This also states the objection period (see below).
    2) We return a numerical ID for this fact, and give you a public key for "yes" and a public key for "no".
    3) One June 1st we pull from a data source and publish the result.
    4) You have a certain amount of time to register an objection by paying us BTC, which triggers a human check. (This helps protect against inaccurate or malicious data from the data source.)
    5) We finalize on a result and publish the relevant private key.
    The result looks like this (human-readable):
    ...or this (machine-readable):

    What we're thinking of doing for Ethereum will be:
    1) We'll have a single data feed contract, along the lines of:
    2) We'll leave the proposition registration process as it is, except that you'll be able to register propositions after they happen, which is currently blocked by our validation code.
    3) We'll initially have an extra bitcoin payment address (and fairly soon the ether equivalent) which you can pay to cause us to send either the binary result or the final value to our data feed contract. This will comprise a small fee for us, plus the equivalent of the amount of ether you think needs to be spent to publish.
    4) When we get a payment we'll publish a key-value pair to the storage of our data feed contract with a hash identifying our registered proposition, and the value of the data we found, or the true/false value of the proposition (depending which you requested).
    5) Initially all of this will be transparent and the data will be public, although in future we may also add some privacy options (probably for a fee) like the ability to register an extra nonce that will be hashed along with the data, so that only the parties can see what the data is for.

    Any suggestions or comments?
  • Ether-LordEther-Lord Member Posts: 7
    sl0anage said:

    @chris613? what would this data feed contract look like?

    I'm wondering the same.
  • koeppelmannkoeppelmann Member Posts: 51
    A few comments:

    To be compatible with Bitcoin writing keys on the blockchain might be a good idea. For ethereum stuff it is not really necessary because other contracts could just directly use the results in the data contract.

    Our oracle contract looks like this:

    data events[]
    data creator

    def init():
    self.creator = msg.sender

    def update_creator(creator_address):
    if msg.sender == self.creator:
    self.creator = creator_address

    def set_winning_outcome(event_hash, outcome):
    if msg.sender == self.creator and not self.events[event_hash]:
    self.events[event_hash] = outcome

    def get_winning_outcome(event_hash):
    return self.events[event_hash]

    So we store a hash of the discretion of the event together with the outcome. By the way: you could modify the "get_winning_outcome" function in a way that it only returns the result if a small payment was made. This data would still be publicly available on the blockchain but not usable for other contracts. (At least if you do not publish a private key)

    By the way - to me it is not clear how this show work what you described in 5) If I have a private nonce/(salt?) - and you publish the hash of the salted hash of the private key - this is not going to help me? But I guess you have something different in mind.
  • edmundedgaredmundedgar Member Posts: 19 ✭✭
    edited June 2015
    @koeppelmann The contract you described was the kind of thing we were thinking of. Although we'd keep publishing keys on the website we wouldn't expect that people would normally use them with Ethereum as it's generally more sensible just to read the data, but they might still be useful to someone.

    The thought behind the private nonce was that when we published an event hash that would have an extra secret appended given only to the person who registers the contract and shared by them with whoever they like, usually their counterparty. So people looking at our data would see:
    Some hash: True
    Some hash: False

    Then people could look at the contract using that data and see that someone was paying someone else if something was true, and it was, but they wouldn't know what the something was. This allows us people to transact privately while still keeping all the facts that we certify publicly available on our website, which we like to do to allow people to check that we're reporting things honestly.

    It occurs to me we might also want a user-supplied data transformation function for the results of numerical feeds, so you couldn't just look at a data point and say, "Well doh, that's USD-JPY".

  • koeppelmannkoeppelmann Member Posts: 51
    Ok thanks for the answers.
    We are currently building a prediction market that should be compatible with external oracles. So let me know if specific code/publishing format exists so that we can assure to be compatible with your oracle as well.
  • edmundedgaredmundedgar Member Posts: 19 ✭✭

    Ok thanks for the answers.
    We are currently building a prediction market that should be compatible with external oracles. So let me know if specific code/publishing format exists so that we can assure to be compatible with your oracle as well.

    Sounds awesome. We haven't worked out the details of how this data should be formatted so if you want to propose something that makes sense from the point of view of somebody building something that uses the data we'd probably do it that way.
  • edmundedgaredmundedgar Member Posts: 19 ✭✭
    edited August 2015
    Just taking another look at this: What are the advantages to the oracle sending the data to a contract and the contract responding when asked, as opposed to the oracle doing everything off-chain and just signing the data, then people who want to use that data pulling it from the oracle's website and providing the data and the signature to their contract?

    I guess one advantage is the ability for the oracle to charge to read data from a contract, if we think that's going to work; Is there anything else?
  • koeppelmannkoeppelmann Member Posts: 51
    @edmundedgar Indeed - the off chain solution might for a lot of use cases be better. Just because the default transactions costs are 0 this way.
    Regarding paying the costs: I am still wondering if another contract can use the data of one contact without using a defined call (that would cost money). I guess this comment of Vitalik suggests it is possible: https://www.reddit.com/r/ethereum/comments/3fmuvz/the_augur_reputation_token_crowdsale_will_begin/ctqjdpj although such a SPV proof could have significant gas costs.
  • edmundedgaredmundedgar Member Posts: 19 ✭✭
    @koppelmann Right, I've never been confident the fee-to-read-from-contract model is going to be enforceable. Even if the SPV proof stuff isn't practical, somebody could make a contract that would "cache" the data from our contract and serve it at a lower cost after the first read...
Sign In or Register to comment.