An organic approach to developing data standards — kwantu

This is the first in a series of blog posts about our work on data standards. I data recovery software free download The intention is to present our work and thinking to a wider audience, learn from you about other work that may connect to this and explore new contexts and partnerships in which we can test these ideas.

Previous posts have covered work we’ve done implementing systems to help manage and monitor development programmes. O review database Since we’ve had the fortune to work on a number of related programmes (in the areas of social accountability and social protection), we’ve also been able to use this work to explore what it means to develop standards for data that similar programmes collect about activities, outputs and indicators.


There is of course a lot of work already done and ongoing on data standards. Database in recovery Some examples are initiatives like IATI, Open Contracting and the Joined Up Data Alliance. Data recovery wizard professional However, when I look for standards related to operational or performance related data I see less progress.

Various sectors and countries have worked to create shared libraries of indicators. Data recovery open source Herb Caudill at DevResults has also made proposals for an indicator standard. Gif database However, it can be time-consuming to facilitate, agree and implement a new standard. Data recovery lifehacker This makes it impractical to invest the time (and by extension money) to develop data standards unless there is a clear return on investment.

From our perspective, indicators are also just the tip of the iceberg. Top 10 data recovery software 2014 Programmes we support collect data on participants, sites, facilities and groups. Database gale They track attendance, satisfaction, feedback and a range of other things. Database life cycle Developing data standards for such a wide range of different needs seems like an impossible task.

Mindful of this we’ve been exploring a more organic approach to developing standards that we feel may work better in this kind of context. Data recovery dallas We call it Self-aware Data Objects (or SDOs).

Before I can explain what we mean by an ‘organic approach’ I need to give some more background. Data recovery usb I’ll start with the building blocks that the approach builds upon.

Let’s explore each of these building blocks through an example. Database 4th normal form Imagine in this case that DFID choose to adopt this approach to manage data collected by programmes that it funds. V database in oracle (We could just as equally use a Government or a network of NGOs as an example).

In this example DFID starts by setting up it’s own registry of data definitions. Data recovery tampa Think of this as an online catalogue. R studio data recovery with crack If you have access you can browse a list of data definitions, add a new one or adopt an existing one to use in your work.

We see this kind of registry as being managed by a data governance team. Database uses Their specific role would depend on which organisation(s) is running the registry. Database history The kind of things they might be responsible for include:

However, we propose one important difference: using a domain specific language to create the data definitions. Database b tree As we’ll see later, this allows a much more decentralised and organic approach to creating and sharing data definitions, which in turn makes the process of agreeing standards much faster. Database optimization But first, what is a domain specific language?

A domain specific language (or DSL) is a programming language designed to be used by domain experts, not just programmers. Data recovery software reviews In this case the DSL we have developed and used in our work is designed to help M&E managers, business analysts (or others that understand data collection needs) create the forms that they need to collect data. Cnet data recovery On one level it’s a tool that lets you build a form from a series of form elements.

This means that data published against this data definition can be validated against the schema. Database systems As the creator of the data definition you can use the schema to enforce business rules that maintain data quality.

The DSL is designed to also create a standard view and edit model. Data recovery for mac This is intended to de-couple data from the application that produces it, making it easy to interact with it without the source application. Data recovery damaged hard drive This has wide ranging implications that I’ll return to in a future post.

Each data definition shares common fields. Database builder We anticipate that these will evolve over time as we understand different user requirements. Data recovery cnet For now they cover information like who created the data and when, who last updated it and when, geographical coordinates and linkages to other data or data definitions.

Since the definition files have been created using a programming language, it is possible to create your own scripts to transform the definition file. Database log horizon In our work we transform the definition files into a JSON file. Data recovery raid However, it could easily be transformed into other file types.

Also, since each data definition is created from the same elements, the task of merging or linking data from different definitions is much simpler.

SDOs can be used to define data at the lowest level at which it will be collected. Database design for mere mortals For example, a workshop attendance register or a group registration form. Database hardening By defining data at the operational level we can better assess it’s quality. Data recovery linux distro Indicators can instead be expressed as a query of the relevant SDO data.

Since SDOs are defined using a common DSL, it becomes possible to make connections between different definitions or data created based on a definition. Data recovery key These can be made explicit, by including a linkage in the definition. Data recovery macbook Or they can be expressed via a query that combines data from different SDOs.

Returning to our fictional example, DFID now has a growing library of data definitions (or SDOs) in their registry. Data recovery los angeles Approved partners can browse this registry and adopt SDOs that they want to use. Database yml The registry keeps track of who adopts an SDO.

If necessary they can modify the SDO, adding additional fields or perhaps translating it into a different language. Database in excel Providing these changes do not conflict with the schema the modified SDO is still compatible with the original SDO. G info database Since the registry manages this adoption process, the SDO versions are linked automatically.

In this way a community of DFID programmes may share their data definitions with each other, adopting, tweaking and using those they consider useful. Database book Through this process of collaboration we see a more emergent and organic way of developing data standards. Q prime database Critically though these are standards in a context only – in this case the context of DFID programmes.

DFID might choose instead (or also) to take a more pro-active approach in some cases. Top 10 data recovery tools A DFID business analyst might assess which data is most needed to report to parliament on the outputs arising from DFID expenditure. Data recovery laptop In this case they might pro-actively define a list of SDOs that provide the necessary data. Data recovery flash drive Programmes might now be required to use these ‘DFID standards’ (at least as a starting point) when selecting SDOs from the register.

While not so much an organic approach it is a way in which a specific donor, government or NGO network can develop it’s own data standards for it’s own purposes.

So far I’ve discussed only four of the key building blocks. Data recovery cost While data standards are important, shared data related to these standards is what we are ultimately interested in.

Since we have used a DSL to create the data definitions the task of aggregating data published to one or more of these definitions becomes much easier. Data recovery galaxy s5 There are three key steps.

First, map your existing data tables to one or more relevant SDOs from the registry. Database key field Second, write a script that transforms your data into the SDO format. Data recovery nashville We use JSON, but since SDOs can be transformed programmatically it could be any format. Data recovery minneapolis Third, integrate with the API to publish your data.

In this way any application can be modified to generate and publish data in this format to a central DFID database. Database 4 net For DFID programmes this means that reporting on activities and outputs can happen in real-time, without the need for a separate PDF or other report.

Data aggregated in this way can be queried – either via the API or using dedicated business intelligence tools. Iphone 6 data recovery software free This makes it possible for different groups to perform their own analysis, limited only by the level of access that they are granted.

While we’ve made a lot of progress on these ideas, there are still many challenges to work out. Database usa reviews I’d welcome your thoughts on these and others that we should consider

Privacy and security: Centralising data in this way is a double edged sword. Easeus data recovery 94fbr Clearly privacy and security implications are of critical importance. Database join Some of the avenues we are exploring are ways of enabling each publisher to encrypt their data. H2 database download They can then choose who they share the encryption key with.

Curation: It’s not hard to picture how quickly the register might fill up with data definitions. H2 database url Clear, well thought through rules and guidelines will be important. Data recovery boot disk Equally important is the need for a team to provide guidance to register users and to curate data definitions already published. 990 database Without this we will quickly see an unusable mess.

Barrier to entry: For smaller organisations with limited capacity, using a DSL may be daunting. Data recovery hard drive cost We need to consider carefully the possibility that this approach adds additional burdens on those least able to bear them. Data recovery knoxville Work on visual tools to create SDOs will certainly help, as will the option of simply adopting SDOs created by others.

Incentives: What are the incentives that will help drive adoption of this approach? In the first instancethose that manage the registry stand to benefit the most – from access to more and better quality data. 7 data recovery keygen However, if the data is accessible it’s not hard to think of how the publishers could derive value too.

For example, less time spent reporting (assuming that donors relax their current requirements), access to data from other related programmes for learning, access to data definitions created by others. H2 database client These and other factors may serve as incentives.

Some of these ideas are well developed and widely used in our work. Dayz database Others are still under development. I phone data recovery Over the last four weeks I’ve had a series of interesting meetings and conversations with people actively involved in the world of data standards. Database 3d Both in relation to open development data and open government data. Yorku database Thanks again to those that took the time to talk, it’s been a great learning experience.

We’d like to hear from people interested in partnering with us to develop these ideas further. O o data recovery If the concepts are validated then our next move will be to launch this as an open source project to leverage wider engagement and adoption.

banner