Rise of the Data Workspace
It's no longer under any doubt that data is essential for business decision-making.
When I was a teenager — just starting out as an engineer — business intelligence (BI) tools were at the heart of collaboration between business and engineering departments.
With their graphical interfaces, BI tools enabled business departments to conduct their own analyses, such as drill-downs. My job involved preparing data and setting up the infrastructure for these BI tools.
This division of labor had engineers handling data preparation, while the business department focused on decision-making using BI tools.
Tool Silos Lead to Business Silos
With the spread of internet and mobile devices, the amount of data created globally has increased a hundredfold over the past decade. It’s said that the number of queries issued on Snowflake now rivals the number of Google searches worldwide.
As businesses needed to leverage more data, the sophistication of analysis methods increased as well, driven by machine learning and AI.
By the mid-2010s organizations began establishing data science teams to handle these complex analyses, bolstered by the rise of Python. Naturally, these teams were also expected to collaborate with business departments as before. Still, a problem arose: the tasks of data science teams couldn't be performed using the BI tools used up to then.
Python is the primary tool for data science, running in local development environments and ideal for data science tasks. However, sharing the process and results with business departments required additional steps.
To share the analysis progress, engineers had to either have members of the business department come to their desks directly or output results as image files for sharing.
To deploy the final product as a dashboard, the program had to be re-implemented in a dashboard-specific application codebase and deployed accordingly.
The simplest way to eliminate these extra steps would be for the business team to run Python on their local machines. However, installing Python and referencing codebases managed by Git is far too high an expectation for the non-engineer business department.
Due to the demand for more advanced analyses, the fragmentation of tools has led to the siloing of business operations, hindering smooth collaboration.
Data Workspaces as Collaboration Hubs in Business
To achieve smooth business collaboration, a shared workspace is needed where both engineers and business departments can make full use of their expertise.
This is where data workspaces which use SQL and Python as common languages are gaining attention as collaborative workspaces. These workspaces are designed for data and enable real-time collaboration, much like Notion or Figma do for text and images, respectively.
Data workspaces free teams from the cumbersome workflows and communication channels associated with data analysis.
Data visualization and automation can be completed within a single tool, eliminating the need to migrate data between different tools, configure settings, or set up environments.
Additionally, analysis results can be easily shared, improving the speed and efficiency of work. Simply sharing a link allows all stakeholders to view the results in real time, with the option for external sharing. Say goodbye to waiting for dashboards to be built or dealing with a flood of Slack requests for data extraction.
In data workspaces, SQL and Python analysis logic can be managed as reusable assets, maximizing the long-term value of data. Moreover, by providing an environment where the business department can directly interact with data, building data marts and reusing data becomes easier, further energizing collaboration.
Thus, data workspaces are more than just data analysis tools—they function as "collaboration hubs" creating valuable assets for both engineers and business teams, enabling sustainable data-driven decision-making.
The Future of AI and Business Collaboration
With generative AI, the barriers to using SQL and Python have dramatically lowered. By utilizing algorithm-driven code generation and explanation, complex analyses can be performed as easily as (and often more easily than) working in Excel or BI tools.
As data workspaces become more widespread, more people will be involved in data-centric business collaboration. AI support will enable business professionals with less technical literacy to participate in complex data analysis.
Moreover, it is anticipated that agent-based AI, which autonomously assists with complex data analysis, will also become more prevalent.
Specialized LLMs/SLMs and agent AIs that perform tasks end-to-end will autonomously handle data processing and insight extraction. Prototypes that deliver such experiences have already been demonstrated.
In this future, the way collaboration happens — including data engineering, AI, and automation — is expected to change. Business hypothesis testing and decision-making no longer require writing even a single line of code.
In the future, many more ideas for business growth can be discussed, tested, and implemented — shortening the runway required for a business to achieve success.
The future where engineers, business departments, and AI collaborate in data workspaces is just around the corner.