– A decentralized data ecosystem (left) that uses Data Mesh architecture (right) for its functional elements
Big Data, Research, Web3

Data Intermediary Networks


In a new research paper published in June 2024, the authors shift the emphasis from ‘data DAOs’ to networks of data intermediaries via data mesh architecture.

In this post, co-author Dr. Kelsie Nabben shares a brief research abstract, and some of the thinking and context behind the research. This project was generously supported by a Filecoin Foundation Developer Grant - Improving Socio-technical Resilience in IPFS: Part 2.

Brief Abstract

data mesh is about distributed data architecture at scale. Our paper argues that “Data Intermediary Networks”, composed of data products built on data mesh infrastructure, can foster cooperation and co-regulation among data product operators, providers, and consumers.

We argue that this leads to an increase in the variety of economically viable data products, along with an expansion of new markets within which those products can be networked together into meshes with the capacity to generate new kinds of value. We then outline the technical, operational, and governance architectures that underpin these networked data economies.

– A decentralized data ecosystem (left) that uses Data Mesh architecture (right) for its functional elements
Figure E — A decentralised data ecosystem (left) that uses Data Mesh architecture (right) for its functional elements. (Sisson, et. al., 2024. See also Deghani, 2022, Fig, 10–11. p. 172).

Read the Full Paper

Sisson, David; Nabben, Kelsie; Ben-Meir, Ilan; and Zargham, Michael. (2024).“Data Mesh Architecture: Interoperability, Co-Operation, and Co-Regulation” SSRN. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4880709.

According to Dietrich Ayala, a long-time IPFS project contributor, these concepts push on some of the intractable conundrums in the IPFS world. In particular, who “owns” content, and how to trace the data lifecycle of who publishes, pays, uses, and cares about it. These questions, which are straightforward in Web2, are incredibly difficult to answer in IPFS and other Web3 architectures.

“This intermediary/mesh work helps us understand the shape of tools to meet user needs in p2p-available environments” Dietrich Ayala

Dr. Kelsie Nabben on Data Architecture Research

Previous Research: I have been thinking about Decentralised Autonomous Organisations (DAOs) as data architecture for a while.

Part 1 of my research on improving resilience in IPFS culminated in a research paper and chapter of my PhD that looked at how social arrangements impact resilience in peer-to-peer networks. See: Nabben, K. (2022). Decentralized Technology in Practice: Social and technical resilience in IPFS.

Some of this began with writing about how DAOs may evolve existing governance architectures for data, such as DAOs as Data Trusts. I then moved to developing practical tools to help designers and developers think through social arrangements surrounding data management to improve resilience, such as the Decentralised Data Governance Pattern Library.

Current Research Context: This recent paper began as an attempt to analyze and operationalize the concept of ‘data DAOs’.

Through desktop research and interviews, the output evolved beyond the rather nascent and mostly theoretical concept of data DAOs to explore the more applicable concept of ‘data meshes’ - undertaken in conjunction with domain experts from BlockScience.

What guided the research was the question of ‘how is value derived from data?’. This led to an exploration of data as a product and data processing pipelines. A linchpin of this work was the use of an Actor Network Theory approach to identify data intermediaries as key actants in the creation of value in data economies.

What’s interesting about this is that a focus on intermediaries seems counterintuitive within a domain of peer-to-peer architectures and ‘disintermediation’. Yet, these actors are critical in the production of value from data, and how they relate to one another determines whether a data ecosystem is centralized or decentralized.

Combined with the technical expertise of my colleagues, this led us to more deeply explore the data mesh architecture and develop the concept presented in the paper of ‘data intermediary networks’.

Enjoy the read! If you have questions, comments, and ideas, you can reach out to Kelsie via her original Substack post. And if you would like to hear more on this research topic subscribe (substack & here!) to catch her forthcoming academic paper on data intermediaries from an ethnographic case study perspective, presented on a panel for the ‘Platform Economies Research Network’ (PERN) conference in New York in April 2024.

You've successfully subscribed to BlockScience Blog
You have successfully subscribed to the BlockScience Blog
Welcome back! You've successfully signed in.
Unable to sign you in. Please try again.
Success! Your account is fully activated, you now have access to all content.
Error! Stripe checkout failed.
Success! Your billing info is updated.
Error! Billing info update failed.