Mistakes, Blunders, Hacking, Coercion and War

cmason · February 2014

I think I'm finally getting the idea of Ethereum and the possibilities it provides. I get the contracts, scripting, enforcement by code, lack of trust, no dependence on third parties, etc. But I think the Autonomous Agent part is the potentially weak area. To use a historical example "Dewey Defeats Truman". For those too young to remember, a major US newspaper proclaimed Dewey won the US Presidency. This was not only completely incorrect, but it caused people to take actions they would not otherwise have taken. The same thing happened more recently when Al Gore was proclaimed the US President before people went to bed, and they woke up to a different President.

So if a news organization or blog or radio station makes a mistake, or an official site (say ESPN) screws up, and an Autonomous Agent relies on this information, contracts can be settled incorrectly, with no recourse. Worse there are often Internet-wide "stories" that develop, that everyone reports on, that are later determined to be wrong. This is just what happens by accident. If malicious actors, with real money at play, are involved, much more active efforts at misinformation could be attempted. This could range from a mass distributed hacking and denial of service attack (change the info where you can, shut it down where you can't) to an insider at Bloomberg manipulating a stock ticker relied upon worldwide for personal profit or political gain.

It's just like social engineering. You never try to break the cryptography, you just present different information in the right places, at the right time, to achieve the result you are looking for. Even if human arbitors are involved, they can be bribed, threatened, blackmailed, extorted or even eliminated. This already happens in the case of theft, fixing sports events, effecting political competitions, or influencing the result in a trial. Add the case of an actual war, and all of this is dramatically escalated.

So, I think the code and protocols, if done right, are probably robust, but it's the interface with the real world where the problem comes in. Because you can't *really* trust anything, not the New York Stock Exchange, not sports scores, not combat casualties, or news reports. They can all be falsified or manipulated, at least to some extent, for some period of time, if there is sufficient motivation.

If I had to summarize my point it would be: "Autonomous Agents are gullible." Is this a concern that has been acknowledged and is there any work towards addressing it?

Thanks very much,

Chris

mlacorte · February 2014

There is a good discussion about this issue going on here: http://forum.ethereum.org/discussion/31/what-would-a-trusted-data-feed-look-like

StephanTual · February 2014

On that note I talked to a guy who works in weather derivatives, and it was interesting to see they trust NASA feeds. More importantly, they have the possibility to reverse transactions should things have been compromised. Would love to hear more from people in that particular vertical on the topic.

edmundedgar · February 2014

I think I agree with most of this. Nobody has 100% reliable methods for getting information to settle contracts, and this will be true whether we use trusted authorities or manage to create some kind of system of consensus witnesses. It's possible to interfere with both the publicly visible outcome and the judgements of people who mediate that information, and this can happen whether those people are trusted arbitors or p2p consensus witnesses.

That said, in the world we currently live in people do actually make contracts like these despite the problems you discuss, so some level of risk is clearly acceptable to the marketplace. The question is then how you mitigate the risk. This is a complex topic, but a few observations:

1) Pulling data feeds from a source that isn't intended to be used for the purposes of settling contracts then trusting it unconditionally is dangerous and potentially irresponsible, because they're not designed or secured with these purposes in mind, and you'd incentivize people to interfere with them. This is why Reality Keys doesn't do this.

2) As with the Dewey vs Truman case, there are trade-offs between accuracy and speed. Users need to be free to choose what the right trade-offs for them are. I suspect people betting on sports would rather get paid quickly at the cost of getting the wrong result from time to time. But if you're buying a house and you don't want the payment to go through until it's been confirmed that ownership has been transferred, the seller shouldn't mind waiting a couple of days until they get their money if that's what it takes to reduce the risk of getting the wrong result from 0.1% to 0.001%.

3) Error and coercion can be mitigated (but not eliminated) with diversity. Users should be able to combine different combinations of arbitors. (Users of Reality Keys can do this already, although we don't yet have any direct competitors so you'd have to use a general-purpose human escrow guy.) Arbitors can build in some jurisdictional and geographical redundancy by having people in semi-autonomous organizations in different jurisdictions, and distributing the keys required to sign any given fact. (Reality Keys isn't yet set up like this, but we're thinking about it.)

4) Bribery and corruption can be discouraged (but not eliminated) with transparency. This is one of the reasons why Reality Keys is set up to make the actual judgements we make 100% public, with a single, shared pair of keys per fact that anyone can check up on. If you pay us to cheat your counter-party, everybody will be able to see us cheating your counter-party, and we'll also have to cheat anybody else sharing the same key.

5) The PKI model with certificate authorities in browsers (or in this case nodes) is nasty, but it may well be better than the alternatives. In particular, the ability to revoke certificates and allow contracts to fall back on some other combination of authorities would be very valuable, especially if revocation was managed properly. However, doing this in a system that's distributed but nevertheless has to achieve consensus creates some interesting problems.

ddink7 · February 2014

It seems like an insurance contract could be set up for this, sort of like title insurance on a house. In the unlikely event that the data feed(s) was/were incorrect, after a certain period of time, one of the parties could send a transaction to the insurance contract, which would then go out and get updated data feeds, and pay the injured party.

I say "after a period of time" because information tends to be more accurate once time has passed and all the data is available (Dewey Defeats Truman could never have happened 24 hours later.) Not only that, but people can only corrupt data for so long...maintaining the corruption of a feed for weeks or months would be difficult and expensive.

w0bb1yBit5 · February 2014

edmundedgar: bravo! Sometimes the crypto-currency community falls into the "give a man a hammer and the whole world looks to him as a nail" syndrome. The blockchain is a magnificent invention, and it opens new vistas of opportunities. But it is not the only tool in the human toolkit and it is not the right tool for all problems. It is not even the right tool for all "trust" problems. And, as you say, there are different levels of trust appropriate for different purposes. The OP says "Autonomous Agents Are Gullible". So are we all. It is my contention that an ethereum contract, with it's ability to evaluate a cryptologically secure signature, has a flexible and useful way of interacting with the off-blockchain world. What combinations of WoT, PKI, redundancies, contingent arbitrators, or insurance contracts one chooses to load onto that signed datum outside of Ethereum is dependent on the human purpose, risk and rewards being protected. That said, it may prove useful to construct some of these additional layers in contracts within ethereum some of the time.

Jasper · March 2014

Still interesting to think about how you would try solve it without the human, even if you shouldnt really trust the script.

w0bb1yBit5 · March 2014

Hmmm. Certificate Transparency project is an interesting development in this area. It seems that there are lots of cool things that can be done with Merkle Trees. MTs are closely related to time stamping, or at least relative time stamping. I am coming to believe that time is a critical component of trust. If you can prevent/discourage lying about the past, trusting the present becomes a matter of reading/interpreting the record.

Howdy, Stranger!

Quick Links

Categories

Mistakes, Blunders, Hacking, Coercion and War

Comments