Creating a collaboration and communication platform to service millions of global users conventionally requires the design and resources of a centralized and proprietary nature, managed by a single entity. Users either have to pay or trade privacy and security for the convenience and ubiquity of access. With recent theft and massive lost of user data across well known social networks and other massive centralized public facing platforms, there is an urgent growing need for an alternative fully decentralized yet secured world-scale collaboration solution.
Open source Rocket.Chat empowers teams and organization from 5 to 500,000 to collaborate and communicate securely and without lost of privacy via Android and iOS apps, web apps, macOS, Windows, or Linux apps. Hundred of thousands of independent servers are deployed, on-premise on in private/public cloud, across the world by groups, organizations, and enterprises with full control over privacy and security.
Gabriel Engel, founder of Rocket.Chat, explores the design challenges for a new federation protocol that can dynamically unite these thousands of de-centralized servers on-demand, forming a world-scale collaboration, communications, and value sharing/exchange fabric for millions of users without the downside risk of user information lost and control.
Introduction:
* Rocket.Chat history - open source MIT licensed server
* Viral community adoption and support that keeps growing
* Easy to extend and customized; popular Javascript on both client and server
* Deployment supported on all server operating system and public / private clouds
* Individual servers scalable from 10s to 100s of thousands of users
* Matured, battle-tested, high performance web and native open source mobile client
* Support for secured data in transit, data at rest, and high grade end-to-end encrypted operation
Thesis:
* Healthy and rapidly growing ecosystem of deployed servers world wide, managed by service provider or by individual group and organizations
* Need to dynamically communicate between servers in a decentralized manner for extended communications and sharing of value - cross focus social networks, social enterprises, B2B relationships, B2C relationships, supply chains ...
* Security, privacy, and decentralized control is preserved while allowing for dynamic functional union of servers in an ad-hoc mesh
Design and Challenges:
* Lack of proven prior-art, solving a problem that has not been effectively solved before
* IRC is both an inspiration and an anti-pattern of what not to do
* Matrix dot org has great ideas but need for decentralized control, privacy and security while scaling is not adequately addressed
* Merging the best ideas from both IRC (the old) and Matrix dot org (the new) and mixing hard real-world business/user requirements to formulate a dynamic federation solution that works for the needs of users next year, in the next 5 years, and beyond
* Optional interoperation and compatibility with both IRC (built in) and Matrix dot org (work in progress) are essential
Takeaways:
* Extreme challenges require innovative engineered solution that preserves the old while transforming to the new
* Learn from mature battle-tested pragmatic implementations and fringe research contemporary design
* Leverage the collective wisdom of a large global open source experts and users community
