Creating an Ethereum Application: Part 1

We decided to build a complete, full-stack demo application using Ethereum & smart contract technology, to learn about all the aspects and challenges there might appear, when working on, if not "full scale", but at least in "full stack" context. This serie of articles is a kind of diary about this undertaking. We'll share our design choices, instrumentation, and essential code samples. All comments and questions are most welcome. The context of the application is sharing economy. The purpose of the application is to enable private customers and small businesess to exchange services of any kind. The application records service contracts to Ethereum blockchain. Furthemore, the services can be charged by using Ether, or by application's own tokens, if chosen so.
Step 1: Designing Data Storage
One of the key difference between a conventional and blockchain-based application is the way data is stored. A blockchain is essentially a data storage, whether using smart contract technology or not (see this nice little eBook about smart contracts). Can we just replace a conventional database with a blockchain? In most cases, this may not be a good idea. As we know, data in a blockchain cannot be updated at all. A workaround for that would be to record the complete update history to the blockchain, but that would raise issues. Primarily, blockchain is an ineffective and expensive way to store data. Secondly, storing a complex set of data to blockchain would become quite difficult, as the data structure cannot be updated while the application evolves (versus SQL ALTER TABLE...). So we end up with a principle that only essential data is stored to the blockchain in some generic way. Say, for example, that we want to store "documents". A generic way would be to define the documents as objects (pseudocode):
Document {id, owner, metadata, data}
"Metadata" can then be a JSON or XML presentation, which makes it easy to store additional fields inside metadata in later versions. To save a lot of space and expences, instead by storing the actual document to "data", it makes sense to store only document hash (hash (Finnish: tiiviste) is a fixed-size, short string, which can be calculated from any amount of data. Later, data can be verified by recalculating the hash. Even the smallest change in the original data yields a completely different hash.). To go even further, should we think that blockchain would store only minimal amount of data: only hashes of both document body and metadata, for example?
Obviously, the final choice is case dependent, but a purist way to store just hashes to the blockchain would make the contents of the blockchain cryptic and heavily dependent of the application outside the blockchain. This would spoil the idea of blockchain as an independent mean of data verification. So if we would set up some guideline, it would be:
Store data to blockchain:
  • in readable format: immutable, essential, small sized data;
  • data which must be agreed by parties;
  • as hash: big data, varying-sized data
Store outside blockchain, e.g. to SQL database

  • mutable and temporary data
  • big data, varying-sized data
  • log and report data

To be more precise, what should we put to the blockchain and what to the SQL database in our case? In an application where anyone can register and provide services, there is a lot of trvial data that needs to be constantly updated, as user information, contact information, categories, notes etc. Only when two or more parties enter an agreement, blockchain becomes relevant. Whatever details are included in the contract, they will be recorded to blockchain, either in readable format or hash. -- Note: to avoid confusion, from nowon we use "contract" in the word's regular meaning, and allways "smart contract" when speaking Ethereum programmatic object.
Step 2: Application Architecture
Which blockchain to use? Here comes a small set of choice criteria:
  • You want or have to use an existing, global blockchain: Ethereum is your only choice. In our case, the target environment for our application is global Ethereum production network. The cost of this choice is that the costs of usage ("gas") must be paid in Ether. Either the one who operates the service must have Ether, or the end users must have it. The actual costs are small, but operating an Ether wallet is not a trivial task yet, especially for ordinary customers.
  • You consider to set up an own blockchain with partners. Now you have a wider range of choices. You may prefer Ethereum technology, or Hyperledger. These are smart contract -capable; that is, your application stores programmatic objects to blockchain. These objects, or smart contracts, are able to execute business logic independently. But if you know for sure that you'll store only data to blockchain, you may use e.g. Multichain, or even Bitcoin technology. A data-only blockchain is easier to work with
  • If setting up an own partner blockchain, you still have to make the setup easy for partners. In this scenario, each partner would run one node ("server") of the network. You may pack a node in a way that a partner can easily deploy an own instance in a cloud service, e.g. in IBM Bluemix.
As said, our target architecture uses the global Ethereum network. We chose to use these technologies:
  • Frontend: web browser/HTML5/SPA application = single-page application, JavaScript intensive, communicates with backend using REST API only.
  • Backend: Node.js , Express REST API, MySQL; web3 Ethereum library -- during test phase, we use Geth Ethereum software to maintain a single-node Ethereum test network, or even lighter, memory-based TestRPC Ethereum client.
The difference between a convential application and Ethereum application is the use of web3 library. It is possible to include the web3 library directly to the browser application, and bybass a separate backend, at least in theory.
The "Browser-only setup" would be attempting, as it would underline the autonomous, de-centralized nature and ideology of Ethereum. However, the user has a bigger responsibility to store his/her own data, keys, Ether, and so on. A more convenient solution would be the "Frontend+Backend" one, which can provide user everything, including Ether administration (wallet 2). However, the idea of Ethereum as a decentralized application environment is compromized. We begin to work on full-stack verion, but we will study the option of browser-only solution, too.
Step 3: Instrumentation What comes to conventional application architecture, we use modern HTML5 + REST API technique with no specialities. What is new to most programmers is to design, test, and deploy Smart contracts, written in Solidity programming language. A recommended development environment is Truffle framework, where you can create a project, create your Solidity contratcs, compile them, generate deployment scripts, and even test them using TestRPC memory-based Ethereum client. The most used Ethereum client Geth provides all Ethereum functionality, but isn't very helpful during the development process. Basically, the application must do these things with Ethereum:
  • connect to Ethereum network, using e.g. http(s) protocol and JSON-RPC calls. The application may operate with plain JSON/RPC, or use a dedicated library, like web3.
  • know the user's Ethereum address. If the application is authorized with a full Ethereum node, e.g. you are running an own Ethereum node to serve the application's needs, you can freely assign new addresses, and transfer Ether to them from another address under the same node. If an address wants to initiate a transaction (move Ether or operate with a smart contract), there must be Ether at the address to cover the gas bill. This is not an issue in a closed partner network, if that is the case.
  • Launch transactions: Ether transfer, smart contract function call, or smart contract deployment.
  • Listen blockchain events. Confirmations of transactions come with delay. Confirmations are expected e.g. to confirm Ether transfer, smart contract deployment, and smart contract data chance. At the confirmation, the application receives important information; e.g. during contract deployment, a dedicated address is created for the contract. The application can access the contract using the address only; the programmatic name of the contract doesn't matter. Some operations are synchronous and won't create a transaction, like creating a new address, or reading data from a contract (at least in local node).
Below are some code examples using Node.js language with current web3 library. It is not essential to study them in order to follow the main storyline. If you want to try Ethereum primitives - Ether transactions, contract deployment and contract function calling - Truffle+testRPC provide you a quick and easy environment. For example, when you launch testRPC, it gives you a set of Etehreum addresses to play with. However, once you stop testRPC, your blockchain is gone. Examples below are tested in Linux using Node 6, Geth (Go Ethereum), and newest Node.js web3 library. Geth is run in test mode, creating a permanent, local blockchain on the disk. A quick setup sequence would be: - install Node.js and Geth. - install web3 library by: npm install web3 Launch Geth background process in most permissive mode: geth --dev --rpc --rpcaddr="" --rpccorsdomain="*" --rpcapi="db,eth,net,web3,personal,web3" --datadir ~/EtherDev/data --ipcpath ~/.ethereum/geth.ipc In a different terminal, connect your local Ethereum network: geth attach Now you are ready to enter Ethereum commands. Note: In order to get your transactions confirmed, mining must be on. You can start it in the Geth console: miner.start() -- in the process terminal, you can see how the block generation propagates. To save disk space, you probably like to say: miner.stop() after work. Also, to do any operations that affect an account, you need to unlock them with password given when creating an account. To run a script (node sample1.js, or nodejs sample1.js), Geth, or alternatively testRPC must be running. Since functions below only display data and won't initiate transactions, mining can be off:
// most functions require a callback function (regular Node.js style)
// so here we have a most general one to suit all cases
function callback(err,result) {
    if (err) console.log("We got err: " + err)
    else     console.log("Success: " + result)

var Web3 = require('web3');
// we have Geth single-node test network at localhost
var web3 = new Web3(new Web3.providers.HttpProvider('http://localhost:8545'));
// let's test some utility function
console.log(web3.utils.toWei(1))  // yields 1000000000000000000
// how much Ether in a known account (in Wei)
// what is our Coinbase address (where mined Ether goes)
// list accounts
In next example, let's assume we have a simple test contract in Solidity language:
pragma solidity ^0.4.14;
contract Somedata {
   string data;   // we store some data and owner
   address owner;

   event dataEvent(string d);  // we signal this event on data change
   function kill() // mean to remove the contract from network   
      if (msg.sender == owner) suicide(owner); 
   function Somedata(string _data) {  // constructor
       owner = msg.sender;
    data = _data;
   function setData(string _data) {  // chance data and signal it
       data = _data;
   function getData() constant returns (string) {  // read data
    return data;
In order to deploy the contract to Ethereum network, it must be compiled. Compilation produces the compiled bytecode as a cryptic string, and an interface description as JSON presentation (ABI). You can get these e.g. using online compiler here. A final code snippet this time is a Node.js program that deploys the contract. Mining must be on:
var Web3 = require('web3');

var web3 = new Web3(new Web3.providers.HttpProvider('http://localhost:8545'));
// here we introduce the ABI interface description
var myContract = new web3.eth.Contract([{"constant":true,"inputs":[],"name":"getData","outputs":[{"name":"","type":"string"}],"payable":false,"type":"function"},{"constant":false,"inputs":[],"name":"kill","outputs":[],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"_data","type":"string"}],"name":"setData","outputs":[],"payable":false,"type":"function"},{"inputs":[{"name":"_data","type":"string"}],"payable":false,"type":"constructor"},{"anonymous":false,"inputs":[{"indexed":false,"name":"d","type":"string"}],"name":"dataEvent","type":"event"}]);
var acc = '0x4eCBc12039603CF66aeCa6A7888feD8955D8B2F9' // use this account
var pass = 'mypass'  // password for account above
web3.personal.unlockAccount(acc,pass)  // unlock account for default time
myContract.deploy({   // the compiled contract code, pasted from the online compiler
    data: '0x6060604052341561000f57600080--terribly-long-string', 
    arguments: ['Initial string']   // constructor parameters
.send({      // who creates the contract and pays the bill
    from: '',
    gas: 1500000,
    gasPrice: '30000000000000'
}, function(error, transactionHash){ 
     if (error) console.log("Error (0) " + error); 
     else console.log("TX hash " + transactionHash); 
.on('error', function(error) { console.log("Deploy error " + error) })
.on('transactionHash', function(transactionHash){ 
    console.log("Deploy transaction hash: " + transactionHash) })
.on('receipt', function(receipt){
    console.log("Success : contract address is " + receipt.contractAddress) // contains the new contract address
.on('confirmation', function(confirmationNumber, receipt){ 
   console.log("Confirmations received: " + confirmationNumber) })
    console.log("Contract instance established, address " + newContractInstance.options.address) // instance with the new contract address
The "on 'confirmation' function()" will remain in memory and report every confirmation it receives from the network. After successfull deployment, we can call the contract's functions using myContract object, as well as listen to contract events. In the next article, we go deeper with design choices and code samples. For example, we'll study the similarities and differences of the programmatic use of blockchain versus conventional database. Again, your feedback is most welcome.
Team Niko Harju, project manager at CGI, blockchain enthusiast Mikko Valjakka, software developer, educator