FINAL WORD
There are probably more than three things you should know , but let ’ s start with these three and go from there . First , it is important to note that AI is real . Yes , it is over-hyped . Yes , entire portfolios are being AI-washed in the same way everything suddenly became a cloud product over a decade ago . But it is real according to the folks who know , which is to say decision makers in our 2024 State of AI Application Strategy research .
While most organisations , 69 % are conducting research on technology and use cases , 43 % say they have implemented AI at scale . That is either generative or predictive .
Somewhat disconcerting is the finding that 47 % of those already implementing AI of some kind have no defined strategy for AI . If we have learned anything from the rush to public cloud , it should be that jumping in without a strategy is going to cause problems down the road .
To help you define that strategy , especially when trying to understand the operational and security implications , we have put together a list of things you should consider .
AI applications are modern applications
It should not need to be said , but let us say it anyway . AI applications are modern applications . While the core of an AI application is the model , there are many other components , inferencing server , data sources , decoders , encoders , that make up an AI application .
These components are typically deployed as modern applications ; that is , they leverage Kubernetes and its constructs for scalability , scheduling , and even security . Because different components have different resource needs , some workloads will benefit from GPU acceleration and others just need plain old CPUs , deployment as a modern application makes the most sense and allows for greater flexibility in ensuring each of the workloads in an AI application is deployed and scaled optimally based on its specific computing needs .
What this means is that AI applications face many of the same challenges as any other modern application . The lessons you have learned from scaling and securing existing modern applications will help you do the same for AI applications .
Leverage existing knowledge and practices for application delivery and security but expand to include approaches that recognise that different components of AI applications may have varying resource needs , such as GPU acceleration for compute-intensive tasks or CPU resources for less compute-intensive workloads .
Modern application deployments allow for flexibility in allocating resources based on the specific requirements of each component , optimising for performance and cost efficiency .
AI applications are different from modern applications
Yes , I know I just hammered home the they are modern applications point but there are differences that impact architecture , operations , and security .
First , AI applications exchange unstructured data . Those prompts have no format , no length or data type requirements , and the eager adoption of multi-modal LLMs only adds to the chaos that is a request . In the sense that most AI applications wrap a prompt and response in a JSON payload , I suppose you could say it is structured , but it is not because the actual payload is , well , undefined .
Second , AI applications communicate almost exclusively with a model via an API . That means bot detection solutions that use human or machine as a base criterion for access are not going to be as helpful . Security services helping to weed out bad bots from good bots are going to be an important part of any AI strategy .
Leaping in without at least a semi-formal strategy for addressing delivery and security challenges is bound to lead to disappointment .
The reliance on APIs is also why , in our 2024 State of Application Strategy research , we found that the top security service planned for protecting AI models is API security .
Interaction patterns for AI applications are often dynamic , variable , and unpredictable . Generally , today ’ s security services watch for anomalies in mouse click and typing rates per page , because the services can infer bot behaviour based on deviations from established human average standards .
That does not work when someone is using a conversational interface , and may type , retype , and submit questions on a highly irregular basis . Given that many security solutions today rely on behavioural analysis , including API security , that means some adjustments will be necessary .
You will need additional security capabilities to properly govern AI applications . Rethink traditional security approaches that may not adequately capture the nuances of conversational interactions . Explore innovative approaches such as realtime monitoring of interaction patterns and adaptive access control mechanisms based on contextual cues .
Recognise the critical role of APIs in facilitating communication with AI models . Invest in robust API security solutions to protect against unauthorised access , data breaches , and malicious attacks .
INTELLIGENT TECH CHANNELS 69