Discover more from David’s Substack
Run Azure AI in your own environment!
Well, not the entirety of Azure just to be clear :)
The madness of AI continues…
Okay, so everyone knows about Microsoft’s large investment in OpenAI. You probably also know Microsoft Azure (and many other cloud vendors) have been offering AI related services for some time. On the Azure front, Microsoft Cognitive Services is one such set of services.
This is all great of course however, you are deciding to send your own data up to the cloud to be processed (be that Azure Cognitive Search, Document Processing or any of the new OpenAI services). You might be fine with that and for a lot of scenarios this would fit the bill.
So, what happens when you want to use say Azure Cognitive Search to organise your own data and provide a better way to search say knowledge base articles, PDF’s and databases? Well, you can certainly use the cloud based Azure Cognitive Search for that and again, in a lot of scenarios, this would be fine. But, if you want to keep your data local/private and not sweep the whole internet (or expose this otherwise) then you can run Azure Cognitive Services locally! Yes, you heard that correctly!
If you have ever heard of Docker (if you haven’t, it is worth checking out not just for developers but consultants/trainers and so many more scenarios) you can use segments of Azure Cognitive Services within docker containers within your environment!! :) Again, you are not going to get the whole of “Azure” in a set of containers, but more targeted services (in my case, docker containers running Azure Search and Azure Document Processing).
Still don’t believe me? Well, here is an example YAML snippet (the infrastructure code to assemble the containers):
Don’t worry if you don’t understand the above code right now, this is one way to describe the docker containers (the units, similar to virtual machines - but not! - that run environments). The key parts are where you see “FORM_RECOGNIZER”. These are the Azure Cognitive Service elements I am using (the form recognizer processes all types of invoice/statements and so on). This is a subset of the wider set of Azure Cognitive Services of course, but you can specify other services in a similar way.
You do have to go through a “gating” process to be approved to use these services locally and of course you still have to pay for the usage :)
The above also applies to the Azure OpenAI Services.
Either way, if you need to localize some of the Azure AI based services, then this is quite a nice option - the only slight downside is trying to run the docker containers on a cheap laptop :) you might need a good slice of RAM and CPU! But just for giggles, you can, and I have, run the Azure Cognitive Form Recognizer containers on my 20gb RAM, Intel Core i7 laptop… just things start to get rather warm quite quickly :)