Use of Artificial Intelligence Calls for Transparency

Challenges related to the use of commercial, proprietary AI systems

Many AI systems are available under proprietary licensing models that provide users with limited rights and information on the key control points of the AI system. There may not be any transparency allowing users to see what type of data has been used in training the AI system, what code and algorithms the AI system runs on, and how the parameters of the AI system have been configured. The same uncertainty follows with the output of the AI system: no or limited information on the training data and operation of the AI system generates ambiguous work products. Companies are also faced with limited options for modifying the AI systems as the applicable proprietary licenses do not generally give access or grant the right to modify the AI system’s source code in any way. As all businesses have their own specific needs and wants while sharing the common need for transparency, proprietary off-the-shelf AI systems may not provide the best value for them.

Today, regulation drives the development of technology and drafting of the related contracts.

Development of own AI systems to bridge the gap for greater transparency

Those technology companies that have the means and capability may want to consider developing their own AI solutions from the ground up in order to get better transparency, control and flexibility on the use of their AI system, especially when the question concerns a critical solution. This allows for tailoring the AI system to their specific needs, and provides for full transparency in the training data as well as functioning of the system. However, developing an AI system from scratch requires an extensive investment in the development work in terms of both time and resources. Also, access to the training data may not be wide enough if only internal pools are used. Regardless of the effort put into developing a customised AI system, there is always uncertainty concerning the results and performance of the system. Therefore, companies with limited resources or no interest in fully developing their own systems must rely on other, more readily available solutions.

Open source AI as an alternative solution

After the release of the most commonly known AI systems, an increasing number of open source AI systems have begun to emerge. The concept of open source AI systems introduces licensing models and information sharing practices that provide users with more freedom, transparency and control. By definition, the open source AI systems’ licensing model provides full access, and thereby full transparency, to the key control points: companies can study how the AI system works, what training data is used and what weight parameters the AI system relies on when generating output. This enables companies to understand the data used for training the systems and makes it possible for them to tailor the systems themselves to their own specific needs without developing the whole system independently. Consequently, companies and other users of open source AI will also understand how the output is generated.

While the Open Source Initiative (OSI) is currently engaged in drafting a definition* of open source AI, there is still no uniform concept of what constitutes an open source AI system. Certain AI systems may provide transparency and modifiability, but their license terms could include restrictions on by whom and for what purposes the systems may be used. This emphasises the importance of understanding the rights and obligations governing the use of any software in general. In order to mitigate the associated risks, companies must establish efficient governance mechanisms for ensuring compliance. (By the way, there are also open source compliance certifications available, such as the OpenChain ISO/IEC 5230 standard.)

Today, regulation drives the development of technology and drafting of the related contracts.

Open source AI as a response to the requirements of the current and upcoming EU regulations

In recent years, the European Union has introduced several regulations that have set increasingly strict requirements for companies either developing or deploying certain technologies. It is evident that innovative companies may not ignore the constantly evolving regulatory landscape. Today, regulation drives the development of technology and drafting of the related contracts. This also applies to AI systems, as they are impacted by various complex requirements governing their training data, permitted use cases, robustness and cyber security. As open source AI systems allow visibility to training data as well as the right to access and modify the source code and other structural elements of the system, companies may independently audit and adjust AI systems to accommodate the requirements of the constantly tightening EU regulations.

AI systems may be offered for use subject to different licensing models. In global business, various territorial regulations impose a number of obligations on developers and deployers alike. The freedom and transparency offered by open source AI may be a part of the solution to ensure regulatory and contractual compliance. Therefore, in addition to seeking different alternatives for AI systems, it is essential that companies identify and adequately respond to the requirements imposed upon them. Simply put, companies must put effort into developing and implementing the necessary governance and compliance processes and mechanisms in their business to meet both license and regulatory requirements. Software development and deployment functions are no longer an exception.

* The Open Source AI Definition – draft v. 0.0.9 – Open Source Initiative