As interest in blockchain and cryptocurrencies heats back up in 2021, there are more blockchains in terms of choice than you can poke a stick at. It just so happens that many of these blockchains are not a beginner or intermediate friendly, requiring tooling and other steps.
If you currently want to build on Ethereum, you need to write smart contracts using Solidity. It’s a whole song and dance. For EOS and other blockchains, it’s a similar story.
In Neblio, things could not be any easier. They provide a REST API and Swagger documented endpoints you can use to query things like balances, token information and more on the Neblio blockchain. After working with the Hive blockchain, I had a need for the ability to stream the blockchain and monitor each block.
While the Neblio REST API does not currently offer the ability to stream blocks, you can interact with the Neblio wallet (either locally or on a server) and make JSON-RPC calls to it. The Neblio wallet works very similarly to the Bitcoin wallet in what kind of calls you can make to it and quite a few other cryptocurrencies which are based on a similar premise.
Why would I want to stream the Neblio blockchain?
In my case, I wanted to monitor transactions on the Neblio blockchain and then I wanted to look for specific NTP1 transactions with metadata inside of them and parse it. Every 30s a new block is produced, a block can contain one or more transactions.
Basically, I want to use Neblio as a database, with the ability to read to and from the blockchain. This will allow you to build your own sidechain, as you can parse each Neblio block, store it into your own database (MongoDB, PostgreSQL, Firebase, etc) and use Neblio for verifying the blocks.
Before we get to the code, you need to update your neblio.conf file.
To find the location of your neblio.conf file, in the wallet click help and then select “Debug Window”
Now, click open for the “Show data directory” and this is where your configuration file will live.
Add the following into your neblio.conf and feel free to change anything, from the username/password as well as the port.
server=1 rpcuser=user rpcpassword=password rpcport=6326
We now have a wallet which can accept JSON-RPC commands, we just need to make sure when we make a request, we supply the user and password values along with the request in plaintext.
Before we proceed, I am assuming you have the latest version of Node.js installed, if not, download it from the official Node.js website here.
Create a new file called streamer.js and add in the following:
const request = require('request');
// Create an async function which queries the local Neblio wallet
// It accepts a method name and an array of parameter values
async function walletRequest(method, params = []) {
return new Promise((resolve, reject) => {
// By default, the Neblio wallet will run on port 6326
let options = {
url: "http://127.0.0.1:6326",
method: "post",
headers:
{
"content-type": "text/plain"
},
auth: {
user: 'user',
pass: 'password'
},
body: JSON.stringify( {
"jsonrpc": "1.0",
"id": "neblio-contracts",
"method":method,
"params": params
})
};
request(options, (error, response, body) => {
if (error) {
reject(error);
} else {
resolve(JSON.parse(body));
}
});
});
}This forms the basis of the code that will make requests to our Neblio wallet. You can change the ID to whatever you want, this name I am using symbolises an app I am building on Neblio, some developers like to supply Date.now() as the ID value.
Add the following code beneath the function we created above. This is where we will stream the Neblio blockchain and will get the current block (including any transactions).
// The last valid block height
let lastBlockNum = 0;
// Utility function we can await to introduce delays into our app
async function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
// Continuously polls the blockchain every second for the latest block height
const stream = setInterval(async () => {
const latestBlockHeight = await getBlockCount();
if (latestBlockHeight) {
lastBlockNum = latestBlockHeight;
}
}, 1000);
// Helper function for getting the latest block height (block number)
async function getBlockCount() {
const response = await walletRequest('getblockcount');
if (response.result) {
return response.result;
}
return null;
}
// Helper function for getting a block by number as well as metadata
async function getBlock(blockNumber) {
return walletRequest('getblockbynumber', [blockNumber, true]);
}
// Function is called for every new block
async function handleBlock(blockNum) {
// If the latest block from the blockchain is greater than or equal to the supplied block
if (lastBlockNum >= blockNum) {
// Load the block
const block = await getBlock(blockNum);
// Display contents of the block
console.log(block);
// If result is not null, it's a valid block
if (block.result) {
console.log(`New block height is ${blockNum} ${block.result.time}`);
// Recrusively call this function again, incrementing the block number
handleBlock(blockNum + 1);
} else {
// We have tried parsing a block that does not exist, call this function recursively
console.error(`Block does not exist`);
handleBlock(blockNum);
}
} else {
// Block isn't ready yet, sleep for 500 milliseconds
await sleep(500);
// Call this function again recrusively, pass block number
handleBlock(blockNum);
}
}
// This function starts the streaming process
// if you do not pass in a block number, always starts from the latest
async function streamNeblio(startingBlock = 0) {
// No starting block specified, we'll get the latest
if (startingBlock === 0) {
// Get latest block height
const latestBlockHeight = await getBlockCount();
// If value is valid, set block height
if (latestBlockHeight) {
lastBlockNum = latestBlockHeight;
}
} else {
// User has supplied a block number to start from
lastBlockNum = startingBlock;
}
console.log(`Starting from block ${lastBlockNum}`);
// Kick off the block loading process
handleBlock(lastBlockNum);
}
// Call streamNeblio to begin "streaming" the Neblio blockchain
streamNeblio();If you have your Neblio wallet running, running the above code would begin to stream the Neblio blockchain. But, how do we get the block data out? We’re streaming the blockchain, but we have no way to get the block itself.
There are a few ways to do this, but I’m going to add in a subscription based callback approach here.
We are going to add in two new functions and variables. First things first, beneath the lastBlockNum variable, add the following:
const subscriptions = []; const ntp1Subscriptions = [];
Now, above the streamNeblio function, let’s add two functions which will register our subscription callback functions for us.
function addSubscription(callback) {
subscriptions.push({ callback });
}
function addNtp1Subscription(callback) {
ntp1Subscriptions.push({ callback });
}The code is self-explanatory. We take the supplied function and push it into an array of subscriptions. These subscriptions will be called from within the handleBlock method.
Now, let’s refactor the handleBlock function to call our callback functions when things change.
// Function is called for every new block
async function handleBlock(blockNum) {
// If the latest block from the blockchain is greater than or equal to the supplied block
if (lastBlockNum >= blockNum) {
// Load the block
const block = await getBlock(blockNum);
// If result is not null, it's a valid block
if (block.result) {
console.log(`New block height is ${blockNum} ${block.result.time}`);
// Loop over all subscriptions and call the supplied callback, passing the block result in
for (const sub of subscriptions) {
sub.callback(block.result);
}
// Does this block contain any ntp1 token transactions?
// we use a reduce to create a new array of ntp1 tokens
const ntp1Transactions = block.result.tx.reduce((acc, value) => {
if (value.ntp1) {
acc.push(value);
}
return acc;
}, []);
// Does this block have any ntp1 transactions?
if (ntp1Transactions.length) {
// Call one or more ntp1 subscriptions, pass the block result as the first argument
// and pass the array of ntp1 transactions as the second argument
for (const sub of ntp1Subscriptions) {
sub.callback(block.result, ntp1Transactions);
}
}
// Recrusively call this function again, incrementing the block number
handleBlock(blockNum + 1);
} else {
// We have tried parsing a block that does not exist, call this function recursively
console.error(`Block does not exist`);
handleBlock(blockNum);
}
} else {
// Block isn't ready yet, sleep for 500 milliseconds
await sleep(500);
// Call this function again recrusively, pass block number
handleBlock(blockNum);
}
}We remove the console.log we had in our handleBlock method and then we add in some loops which iterate over our subscriptions and call them. For ntp1 transactions, we loop over the tx array and if the boolean ntp1 is truthy, we create an array of transactions.
Putting it all together
The complete result of our streamer looks like this.
const request = require('request');
async function walletRequest(method, params = []) {
return new Promise((resolve, reject) => {
let options = {
url: "http://127.0.0.1:6326",
method: "post",
headers:
{
"content-type": "text/plain"
},
auth: {
user: 'user',
pass: 'password'
},
body: JSON.stringify( {
"jsonrpc": "1.0",
"id": "neblio-contracts",
"method":method,
"params": params
})
};
request(options, (error, response, body) => {
if (error) {
reject(error);
} else {
resolve(JSON.parse(body));
}
});
});
}
let lastBlockNum = 0;
const subscriptions = [];
const ntp1Subscriptions = [];
async function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
const stream = setInterval(async () => {
const latestBlockHeight = await getBlockCount();
if (latestBlockHeight) {
lastBlockNum = latestBlockHeight;
}
}, 1000);
async function getBlockCount() {
const response = await walletRequest('getblockcount');
if (response.result) {
return response.result;
}
return null;
}
async function getBlock(blockNumber) {
return walletRequest('getblockbynumber', [blockNumber, true]);
}
async function handleBlock(blockNum) {
if (lastBlockNum >= blockNum) {
const block = await getBlock(blockNum);
if (block.result) {
console.log(`New block height is ${blockNum} ${block.result.time}`);
// Loop over all subscriptions and call the supplied callback, passing the block result in
for (const sub of subscriptions) {
sub.callback(block.result);
}
// Does this block contain any ntp1 token transactions?
// we use a reduce to create a new array of ntp1 tokens
const ntp1Transactions = block.result.tx.reduce((acc, value) => {
if (value.ntp1) {
acc.push(value);
}
return acc;
}, []);
// Does this block have any ntp1 transactions?
if (ntp1Transactions.length) {
// Call one or more ntp1 subscriptions, pass the block result as the first argument
// and pass the array of ntp1 transactions as the second argument
for (const sub of ntp1Subscriptions) {
sub.callback(block.result, ntp1Transactions);
}
}
handleBlock(blockNum + 1);
} else {
console.error(`Block does not exist`);
handleBlock(blockNum);
}
} else {
await sleep(500);
handleBlock(blockNum);
}
}
function addSubscription(callback) {
subscriptions.push({ callback });
}
function addNtp1Subscription(callback) {
ntp1Subscriptions.push({ callback });
}
async function streamNeblio(startingBlock = 0) {
if (startingBlock === 0) {
const latestBlockHeight = await getBlockCount();
if (latestBlockHeight) {
lastBlockNum = latestBlockHeight;
}
} else {
lastBlockNum = startingBlock;
}
console.log(`Starting from block ${lastBlockNum}`);
handleBlock(lastBlockNum);
}Is it the prettiest? Definitely not. Does it need more work before using it in production? Absolutely. But, for what started out as a proof of concept, it works beautiful.
Running it
Now we have a function streamer, we can run it.
Stream the Neblio blockchain and call a function for every new block
Every new block loaded, we will console log the output of it.
addSubscription(block => {
console.log(block);
});
streamNeblio();Running the above should result in seeing blocks being console logged in your console that looks similar to the following. Every 30s you should see a new block printed out.
Get a specific block containing one or more NTP1 transactions
We tell the streamer to start at a specific block height which contains an NTP1 transaction I put onto the chain. The ntp1 subscription will fire and the ntp1 transactions console logged for this particular block.
addNtp1Subscription((block, txs) => {
console.log(txs);
});
streamNeblio(2699709);This is the result you should see in your console.
Get a specific block containing one or more NTP1 transactions (and parse the userData)
The above block contains an NTP1 transaction I made, this transaction also contains some metadata. Here is how you would get the metadata for each transaction.
addNtp1Subscription((block, txs) => {
for (const tx of txs) {
const { metadataOfUtxos: { userData: { meta } } } = tx;
console.log(meta);
}
});
streamNeblio(2699709);This is the result you should see in your console from the above command.
Conclusion
This blog post shows you how you can effectively use the Neblio blockchain as a database. I am currently in the process of turning the above into a reusable library you can use to build sidechain-esque applications that leverage Neblio behind the scenes but have a layer on-top.
Donations
If this blog post helped you and you’re feeling generous, I am accepting Neblio donations to my Neblio wallet here: NQvjvfzRh2DS5CfeebSNY4dByMwrRQGTYX — Any support will go towards building the above into the planned sidechain library (which will have database support and adapters for different databases, including Firebase).
