article archive

Failing OmniLedger login

Connection Failures

There are some common reasons why the OmniLedger login can fail. So far we have seen the following ones:

  • Corporate firewalls
  • Private browser settings

If you have a corporate firewall that blocks access to non-standard ports, the only thing you can do is to remove the VPN. This is often enough to let OmniLedger login work, at least if you’re not in your office.

Concerning the private browser settings, the problem is the following: OmniLedger login doesn’t rely on passwords, but rather it relies on a secret private key. This private key is stored in your browser. Unfortunately storing data is often abused, so some browsers are set to private navigation all the times. This means that the browser will not store the private key. So the OmniLedger login cannot work! Here are some settings how you can overcome that:

  • Firefox
    • Use a normal window instead of a “Private window”
    • In the preferences of Firefox, under “Privacy and Security”, the “History” setting should be on “Remember History”
  • Chrome
    • Use a normal window instead of an “Incognito window”
    • In the settings of Chrome, under “Privacy and Security”, make sure that
      • “Block all cookies” is NOT selected
      • “Clear cookies and site data when you quit chrome” is INACTIVE
  • Safari
    • Use a normal window instead of a “Private window”
    • In the preferences of Safari, under “Privacy”, make sure that “Block all cookies” is DESELECTED
  • iOS
    • In the Settings, under “Safari”, remove “Block all Cookies”

Maturity Evaluations

Maturity Evaluation

The C4DT/Factory works to bring EPFL’s great ideas to our partners. We do so by working on the existing code created in the labs. Because most of the code written in the labs has the goal to create graphs in a paper, it is not directly usable in another project. For this reason we created the Maturity Evaluation. In short, this is a simple Google Spreadsheet. with different evaluations to fill out. At the end we see how the code can be improved. The Maturity Evaluation also gives us an indication of the overall quality of the code: Prototype, Intermediate, or Mature.

The showcase.c4dt.org includes the result of the Maturity Evaluations we did so far. Every lab that is associated. with C4DT can ask us to evaluate their code. Then we will add it to the showcase.

The Maturity Evaluation has the following sections:

  • Code – which evaluates the readability and documentation of the code
  • Culture – everything around the code, including setup of the github account, pull requests, issues, and many more
  • Context – paper references, integration with other code

You can get a copy of the spreadsheet here: Maturity Evaluation V0.5 There is also a document explaining the different evaluations and what they refer to: Maturity Evaluation Explainer V0.5

If you have questions or comments, please don’t hesitate to get in contact with me: linus.gasser@epfl.ch

TLDR for “Managing Technical Quality”

Some notes from https://lethain.com/managing-technical-quality/ – most of the comments are copy/pasted, so ‘I’ is the author… In italic some more C4DT-specific notes.

  • Technical quality is a long-term game. There’s no such thing as winning, only learning and earning the chance to keep playing.
  • Do the quick stuff first!

Hot Spots

  • Before defining new processes, measure the problem at hand, identify where the bulk of the issue occurs, and focus on precisely that area.

Maturity Evaluation application, work on red boxes of column I

Best Practices

When you’re rolling out a new practice, remember that a good process is evolved rather than mandated. The handful that I’ve found most helpful to adopt early are version control, trunk-based development, CI/CD, and production observability (including developers on-call for the systems they write), and working in small, atomic changes.

Maturity Evaluation, yellow boxes, needs buy-in from the lab and regular follow-ups.

Leverage Points

  • As you look at how software changes over time, there are a small handful of places where extra investment preserves quality over time, both by preventing gross quality failures and reducing the cost of future quality investments.
  • I call those quality leverage points, and the three most impactful points are interfaces, stateful systems, and data models.

This is probably the last point that we can influence as the C4DT Factory. This requires that the lab is OK with us working on the software artefact in question.

Technical Vectors

One sure-fire solution to align technical direction is to route all related decisions to the same person with Architect somewhere in their title. This works well, but is challenging to scale.

More decentralized approaches include:

  • Give direct feedback.
  • Articulate your visions and strategies
  • Encapsulate your approach in your workflows and tooling
  • Train new team members during their onboarding

Measure Technical Quality

My experience is that it is possible to usefully measure codebase quality, and it comes down to developing an extremely precise definition of quality.

After you’ve developed the definition, this is an area where instrumentation can be genuinely challenging, and instrumentation is a requirement for useful metrics.

Technical Quality Team

When spinning up and operating one of these teams, some fundamentals of success are:

  • Trust metrics over intuition
  • Rotate with developers

Quality Program

Operating organizational programs is a broad topic about which much has been written

  • Identify a program sponsor
  • Identify program goals for every impacted team and a clear path for them to accomplish those goals
  • Build the tools and documentation to support teams towards their goals

Recover data on ByzCoin

How to get data back on ByzCoin

OK, now it happened – I lost my addressbook on ByzCoin. As we’re using the Partner-login internally, I have a list of all accounts I created on the blockchain. Currently it’s not encrypted using Calypso yet, work in progress. So my account on Byzcoin has a slice of all IDs of all accounts I created. This is useful when a user loses his account and wants it recovered. As long as the user did not remove his trust in me, I can do that using a special recovery-rule in DARCs.

Problem is that I deleted my addressbook 🙁 It happened when I was working on reviving an old mobile app for use on the blockchain: Proof of Personhood. The nice thing is that you can link the account on the blockchain with the app. The bad thing is that the app deletes the addressbook! Well, fixed that in the meantime. But, too late.

But ByzCoin is a blockchain, so the information is still stored somewhere in the blocks, so let’s recover it.

The ByzCoin Explorer

I wanted to use https://stackblitz.io/github/c4dt/ol-explorer, but for some bitrotting this does not work anymore. The CEO of stackblitz was quite surprised, but, well, it still doesn’t work. So installing it locally and working locally with it:

git clone https://github.com/c4dt/ol-explorer
cd ol-explorer
npm ci
npm start

This is some hacky way to interact with ByzCoin: all the needed libraries are loaded, the test-network of DEDIS is preconfigured, and you can connect to your account on byzcoin. For programming, you can use your preferred IDE with all the automatic completion and documentation that is available. Would be very nice to be able to use stackblitz, so you would not even need to install anything. But welll…

OK, first thing is to connect to your user on byzcoin: supposing you have an account, on the omniledger-demo, chose Add Device and enter some name. Once the device has been added to your account, copy the link, and on localhost:4200 (if your npm start is still running) click on Attach User and copy the url from the new device.

Getting old Information

Now that the blockchain explorer is linked to our account, we have write-access to it on ByzCoin, so we need to be careful. If we bust it, there might be no way back at all… In the ngOnInit method in src/app/app.component.ts, add the following after the this.logPP…. line.

const versions = await this.bcs.bc.getAllInstanceVersions(this.bcs.user.credStructBS.id);
for (const version of versions){
    this.log(`Version ${version.statechange.version} at block ${version.blockindex} with length ${version.statechange.value.length}`);
}

This will ask one of the nodes to return all known instance-versions and print them out. Unfortunately, this is not a reliable method. The reason is that the nodes do this on a best-effort basis. They don’t need to hold the whole blockchain, as currently about 25 blocks out of the 100’000 available are enough to go from the genesis-block to the latest block. So they always hold the latest version of an instance, but not necessarily all versions. So it might take some F5 reloadings to get old data from a random node…

As it prints the version-number and the length of the data, I simply looked at the place where the length of the data all of a sudden decreased – this is where my addressbook got deleted!

17:06:48 -> 
Version 778 at block 100218 with length 2961

17:06:48 -> 
Version 777 at block 100214 with length 2959

17:06:48 -> 
Version 776 at block 100213 with length 4913

17:06:48 -> 
Version 775 at block 100211 with length 4913

So the last good version of my credential instance seems to be version 776.

Setting new data

This part is not for the faint of the heart. It overwrites the data on the blockchain, so if you remove for example all access delegations, nobody will be able to recover anything from your account. Which is very nice if you want to delete it, not so nice if you want to fix things.

Before I overwrote the data, I actually had a look at it if it made sense. The following code gets version 776 from my instance, the one with the bigger data, and prints out the addressbook. The addressbook is stored in the Attribute of the credential under 1-public/contactsBuf as an array of bytes:

// Get the old version of the credential instance
const contactInstance = await this.bcs.bc.getInstanceVersion(this.bcs.user.credStructBS.id, Long.fromNumber(776));
const credStruct = CredentialStruct.decode(contactInstance.statechange.value);
// Extract the addressbook and display it
const contactsBuf = credStruct.getAttribute("1-public", "contactsBuf");
this.log(contactsBuf);

That looked good – it was a long stream of bytes, which I hoped was my addressbook. To play it safe, I could’ve used the OL-demo under Contacts and copy 64 hex-bytes at a time using Link Contact. But there were many contacts, so I double-checked the following and ran it:

this.bcs.user.executeTransactions((tx) => {
    this.bcs.user.credStructBS.credPublic.contacts.setInstanceSet(tx, new InstanceSet(contactsBuf));
})

This creates a new transaction and executes it. The dynacred-library allows for updating the structures by passing this transaction tx variable to specific methods. Chaining multiple instructions is done by calling multiple methods in the callback of the executeTransactions method. The transaction will be signed and sent to ByzCoin, and if all instructions are valid, the global state will be updated with the new data:

MY ADDRESSBOOK IS BACK!

Recovering an account

So, in fact this was great, as somebody just asked me to recover his account. With my addressbook back, I could search his account, and, as he still trusts me, click on Recover. This is similar to adding a new device to the account, which creates an URL that can be used to link a browser to the account. So I sent this URL over email. Once clicked on the URL, it will connect to demo.c4dt.org/omniledger and store the private key in the browser. Then the person should be ready to use it. Or not, if there is a corporate firewall 🙁

Coding Guidelines @ EPFL

There is a variety of coding guidelines at EPFL. Some focuses on a lab’s preferred language and its some specific tools, some are more on project handling and general git usage. Most are a mix of the above.

Sadly, theses guidelines aren’t scrupulously followed, aren’t usually automatically checked and some are even unknown to the originating lab. The aim of this article is to raise awareness of theses documents. First, by having a list of the existing ones, then to work with the labs to improve its usage and usefulness. Hopefully, by referencing theses rules regularly and by providing easy ways to implement it, general code quality will improve. It will be linked with the Maturity Evaluation.

Here goes the list of currently known documents and some comments to quickly grasp each. If you know some more, please contact Valérian Rousset.

  • LDS
    • overall very good guidelines, from readability to content
    • coding guidelines: formatting, testing, documentation
    • project handling guidelines: how to handle a repository and cooperation in a team
    • git/github: branch handling, CODEOWNERS to auto assign review, do small commits/PRs
    • golang: language specific tips & good usage, good links to libraries
  • DEDIS
    • quite small with code examples, very opiniated
    • tests: naming, helpers at the end, test first “happy path” then errors
    • errors: be verbose
    • golang/comments: syntax for implementation, 80chars
    • golang/misc: use empty lines, constructor starts with New, do not use named return
  • SPRING
    • quite small with some libraries & code examples, into a larger “student onboarding” document
    • python/doc: state input & output with types
    • python/code: format using black
    • python/tests: unit tests (one per function), separated by module, use pytest
  • MLO
    • python/code: format with blake, use Makefile with checker
    • git/github: use semantic versioning branch (v2), PR must pass tests & lint, maintainer approved
  • LSIR
    • git/github: use git-flow
    • python/code: format with flake8