The software company that created FT Mercati Go to ftmercati.com

Guide · Technology for decision makers

AI in the cloud or on-premise: how to choose

AI models in the cloud or in-house (on-premise, offline): where the data goes, how you pay, what quality you get, who maintains them, which rules apply.

By FT Support Updated on 17-minute read

The short answer

An AI model is used in two ways. In the cloud, that is, online: it runs in the provider's data centres and you pay by usage or subscription. On-premise, that is, locally or offline: an open-weight model runs on the company's computers and the data stays inside. The cloud gives the most capable models with no investment; on-premise gives control and works without a network, but needs hardware, maintenance and smaller models. Often you need both.

  • In the cloud: more capable models, no investment, usage-based cost. The data passes through a provider, under a contract.
  • On-premise: the data stays in the company and you can work even without the internet. You need hardware, maintenance and smaller models.
  • The real risk to data is personal accounts, not the cloud under a company contract.
  • In the cloud you pay by usage; on-premise you pay for the machine even when it is idle: which costs less depends on the volume and regularity of the work.
  • Keeping data in-house does not automatically make you compliant: the GDPR and the AI Act apply in both cases.
  • The most common choice is mixed: on-premise for what is confidential, the cloud for the hardest tasks.

Note. Models and contract terms are those published by the providers as of 30 September 2026 and they change often: before deciding, check them again on the official pages listed at the end. This guide gives no prices and no comparisons in figures: the sums are done case by case. The sections on the rules are for information only and are not a substitute for legal advice.

Online and offline: where the model runs

A language model, the engine of the artificial intelligence (AI) this guide is about, is a very large program. The first choice to make is where to run it.

In the cloud, that is, online. The model sits in a provider’s data centres. The company sends it the text to work on and receives the answer, through an application such as ChatGPT, Claude, Copilot or Gemini, or through the API, that is, the connection that allows a company program to use the model.

On-premise, that is, locally. The model is downloaded and installed on one of the company’s computers or servers. The processing takes place there: the data does not leave the company, and the system also works offline, without an internet connection.

Only open-weight models can be installed on-premise: those for which the maker publishes the file containing the parameters. The most capable models from the big providers are not open-weight, and can be used only in the cloud.

The choice is not new. According to ISTAT (Italy’s national statistics institute), three in four businesses in Italy (75.6%) already buy cloud services: email, documents and often the ERP (the company’s management software) have been outside the company for years. With AI the question comes back, because it is precisely the most sensitive information that is entrusted to the model.

Getting the words straight

The termWhat it means
In the cloud, onlineThe model runs at the provider’s and is used over the internet
On-premise, local, self-hostedThe model runs on the company’s machines
OfflineIt works without a connection: this is possible only on-premise
On-deviceThe model runs on the individual computer or phone of the person using it
Private AIA marketing term: it can mean an on-premise model or a cloud service reserved for a single customer. Always ask which of the two
Open-weightThe model’s parameters can be downloaded. The licence does not always allow every use
Open sourceIn the strict sense: it can be used, modified and redistributed freely, including for commercial purposes
APIThe connection through which a program uses a model in the cloud, paying by usage

Models in the cloud

Three ways in, with different rules

Saying “we use AI in the cloud” is not enough, because there are three ways in, and for the data that changes everything.

QuestionPersonal accountPlan for organisations or APIOn-premise model
Is the data used to train the models?Usually yes, unless you opt outNo, by defaultNo
Where it is processedWherever the provider decidesWhere the contract says; you can often choose EuropeIn the company
How long it stays with the providerUntil it is deleted; in some cases, yearsOn the APIs, usually up to 30 days; in the applications for organisations, the administrator decidesNever
Who has a contract with the providerThe individual employeeThe companyNo one

The biggest risk to company data is personal accounts, not the cloud in itself. Anyone who pastes a contract into an assistant used with their own private account hands it over on terms the company has not chosen. This is why written rules are needed: there is an example in the guide Company AI policy: usage rules and an example.

What the providers’ terms say

As of September 2026, according to each provider’s official pages:

  • OpenAI. Data sent to the API is not used to train the models; the abuse monitoring logs are kept for up to 30 days. Processing in Europe is available to eligible organisations, with a 10% surcharge on recent models.
  • Anthropic. In the commercial products, no training by default; on the API, requests and responses are deleted within 30 days. The direct API does not offer a Europe-only option and stores the data in the United States: to stay within the European Union, you use the same models through the Amazon or Google clouds.
  • Microsoft Azure. Requests and responses are not available to OpenAI and do not train the models; with the European configuration, processing stays within the member states of the Union.
  • Google Cloud. No training without the customer’s permission; the European service endpoint guarantees processing within the member states, the global one does not.
  • Amazon Bedrock. By default it does not store requests and responses, with declared exceptions for some models; to stay within the European Union there are geographic profiles.
  • Mistral. A French provider: the data is hosted in the European Union by default; on the API, retention is 30 days.

With personal accounts the rules are different. In the consumer versions of ChatGPT, content may be used for training unless the option is switched off. In Claude it is the user who chooses, and if they consent the data can be kept for up to five years. In the Gemini app, the use of conversations to train the models depends on a specific setting, separate from the history; some conversations may be read by reviewers in any case.

What it gives and what it asks for

The cloud gives you the most capable models, kept up to date by the provider, with nothing to buy and a cost that follows usage. It asks for a reliable connection, a carefully read contract and a willingness to let the data pass through a third party.

On-premise models

Which models

As of September 2026, the main families of open-weight models are Google’s Gemma, the Mistral models, Alibaba’s Qwen, Microsoft’s Phi, IBM’s Granite, OpenAI’s gpt-oss, DeepSeek (the block ordered in January 2025 by the Garante, Italy’s data protection authority, concerned the online service, not the downloaded weights) and Meta’s Llama. There are also models built for Italian, such as Minerva from Sapienza University of Rome.

Two things to know before downloading one.

A “local ChatGPT” does not exist. ChatGPT is a service. What you install is an open-weight model, which may come from OpenAI (the gpt-oss family) or from others: a different model from that of the online service, and generally a less capable one.

Open does not mean free of conditions. Many models have licences that allow any use, including commercial use. Others do not: the Llama 4 licence, for example, does not grant the rights to the multimodal models to companies whose principal place of business is in the European Union, and some licences exclude companies above a certain revenue threshold. Read the licence before choosing the model.

What machine you need

Memory matters more than speed: the whole model has to fit into it. The rule of thumb is about 0.6 GB for every billion parameters, with the model compressed to 4 bits, plus the space for the text it is working on. More requests at the same time and more text per request increase the memory needed.

Class of machineExamplesMemoryModels it can run
High-spec office computerMac mini with 24 GB, or PC with a 16 GB graphics card16-24 GBUp to about 20 billion parameters
WorkstationPC with a 32 GB graphics card, or Mac Studio with 36 GB32-36 GBAbout 30 billion
High-memory machineMini PC with 128 GB of unified memory, Mac Studio with 96 GB, or a 96 GB professional graphics card96-128 GBAbout 120 billion
Data centre serverEight cards, from 640 to over 1,400 GB of memory, 10-14 kW of powerOver 600 GBThe largest open models

An office machine handles a few requests at a time well; for dozens of people using it at once, more memory is needed.

What it gives and what it asks for

On-premise gives you three things: the data does not leave the company, the system works without a network, and the cost is fixed and does not depend on how much you use it. It asks for an investment, someone to maintain it and the acceptance of smaller models.

The middle ground

Between “everything from the provider” and “everything in-house” there are intermediate solutions, often the most sensible ones.

  • Cloud with processing in Europe. The same models from the big providers, with a contractual guarantee that requests and responses stay within the member states.
  • A European provider. Cloud models from companies based in the Union, with the data hosted in Europe by default.
  • Open models in a European cloud. An open-weight model run on rented machines: the control you have over a model of your own, without buying hardware.
  • Hybrid. On-premise, whatever touches confidential data or has to work without a network; in the cloud, the difficult tasks that contain nothing sensitive. A program routes the requests according to written rules.

The data: which, before where

Before asking where to run the model, it is worth asking which data would pass through it.

  • Personal data of customers, suppliers and employees. Here the GDPR applies, in both cases.
  • Trade secrets: technical drawings, bills of materials, recipes, process parameters. What protects them is not the GDPR but the confidentiality the company manages to guarantee.
  • Other people’s data: customers’ drawings and specifications, often covered by non-disclosure agreements. In some supply chains these agreements forbid taking the customer’s data onto external services. If the contract forbids it, the choice is already made.
  • Sensitive commercial data: purchase prices, suppliers’ terms, price formulas in contracts, hedging positions on metal.
  • Public or not very sensitive data: market prices, regulations, catalogues, manuals. For these the cloud poses no problems.

A company that processes metals usually has all five categories. It is the typical case in which a mixed choice makes sense: public prices can go anywhere; purchasing terms and customers’ drawings cannot.

The rules: what changes and what does not

GDPR. With a cloud provider that processes personal data on the company’s behalf, you need a contract appointing it as data processor, as provided for in Article 28. If the data goes outside the European Union, the rules on transfers apply. For the United States there is currently an adequacy decision, the Data Privacy Framework: the General Court of the European Union dismissed the action to annul it in September 2025, and the appeal before the Court of Justice is still pending. It applies only to the US companies that have joined it, and it does not replace the Article 28 contract.

Foreign law. A US provider remains subject to the laws of its own country even when it processes the data in Europe. As early as 2019, the European data protection authorities pointed out the possible conflict between the US CLOUD Act and the GDPR. It is one of the reasons why some companies prefer European providers, or on-premise, for their most sensitive data.

On-premise does not make you compliant automatically. Keeping the data in-house removes the handover to a third party. What remains is the legal basis for the processing, the privacy notices, security, retention periods and people’s rights.

AI Act. Whoever uses an AI system under its own authority is a deployer, whether the model runs in the cloud or in-house. The obligations are the same: measures for staff AI literacy, transparency, and heavier rules only for high-risk uses. Anyone who modifies an open model in depth may take on the obligations of a provider; the European Commission’s guidelines, however, set a high threshold, which a normal adaptation does not usually reach. We discuss this in the guide AI Act: what a manufacturing company must do.

NIS2. Medium-sized and large companies in some manufacturing sectors, including machinery, electrical equipment, electronics and vehicles, fall under the cybersecurity rules; metallurgy and metal products are not on the list, but a company that supplies a NIS entity may be involved as a supplier. The mandatory measures include supply chain security: a cloud AI service is a supplier to be assessed like any other. But a machine in the company also has to be protected and kept up to date.

How you pay: the cost items, without figures

Those who sell hardware work out how many months it takes for the machine to “pay for itself”; those who sell cloud services do the opposite sum. The difference lies in the assumptions: how many requests, how much text per request, which model, how many people. This is why we do not publish a comparison in figures: we do it on the actual case, with the company’s own numbers. The cost items, however, are always these.

In the cloud, by usage. You pay per token, that is, per fragment of text read and written by the model: a page is worth roughly a thousand tokens. The price per token changes with the size of the model, and small models cost a fraction of large ones; text already read in earlier requests costs less. The bill follows usage: small when usage is small.

In the cloud, by subscription. You pay per user per month. It includes the ready-made application, but not the connection to the company’s documents and systems; it grows with the number of people, not with usage.

On-premise. You pay for the machine once, to be depreciated over a few years; for the energy, which for a workstation is a small item; and for maintenance, the item that gets forgotten: someone has to update the model and the software, monitor the load, and step in if it stops. The cost does not depend on how much you use it.

What follows from this:

  • With few, variable requests it is usage that counts; with high, steady volumes it is the fixed cost. The point at which on-premise becomes worthwhile depends on the volume, on the model chosen and on who does the maintenance: if a small model is enough for that task, small models cost little in the cloud too.
  • The comparison must be made at equal quality. An on-premise model with 30 billion parameters does not do the same things as a mid-range model in the cloud.
  • The per-user subscription is the route that requires no technical work at all, and the one that grows with the number of people.

Three items are missing from these sums, and they weigh more than the model: the work of connecting the system to the company’s documents and programs, the checks and staff training. They are the same whichever route you take. We have described them in the guide What an AI project costs: what it depends on.

Quality: how far apart they are

There are two gaps, and it is best not to confuse them.

The first is between the best open models and the best closed models. It is small. According to Stanford University’s AI Index 2026, in March 2026 the best closed model was ahead of the best open model by 3.3%, in a ranking based on users’ judgement. But those open models are very large and run only on data centre servers.

The second is between the models that fit on an office machine and the models in the cloud. It is larger, and it depends on the task.

  • Where a local model is usually enough: summarising, classifying, translating, extracting data from documents with a recurring structure, answering questions on short texts.
  • Where the cloud stays ahead: multi-step reasoning, very long documents, in our experience polished writing in Italian, AI agents that use tools and have to complete a task on their own.

Agents deserve a note. They work with local models too, but they need more memory, because they work on a lot of text at a time, and, in our experience, they make mistakes more often in choosing and using tools. The same AI Index notes that even the best agents fail about one attempt in three in structured tests: with smaller models, checks and confirmations matter even more.

The serious way to decide is to test your own task: twenty or thirty real cases, run on the two models, with someone judging the results. A local model can be tried out on rented hardware before you buy a machine.

Without the internet, literally

There are situations in which there is no connection, or there must not be one: a production department isolated from the network for security reasons, a construction site, a line that cannot stop if the connection drops. Here on-premise is the only route, and there is no comparison to make.

The reverse also holds: a cloud system stops if the line goes down. For processes that cannot wait, you have to plan for what happens in that case.

How to choose: six questions

  1. Which data would pass through the model? If it is confidential or covered by agreements with customers, the choice starts from there.
  2. Where does it have to work? In an office with a network connection, or in a department that has none.
  3. How difficult is the task? Classifying documents is one thing; having an agent work inside the ERP is another.
  4. How many requests, and how regular? Few and variable favour paying by usage; many and steady favour the fixed cost.
  5. Who maintains it? If there is no one in the company, maintenance has to be bought in.
  6. How do you get out? If in a year’s time the terms or the models change, what has to be redone to change course.
SituationThe choice that usually makes sense
An office that wants an assistant on procedures, quotes and manualsCloud with a plan for organisations or the API, where possible with processing in Europe
Drawings and specifications from customers with strict non-disclosure agreementsOn-premise
A department with no connectionOn-premise
An agent working inside the ERP on multi-step tasksCloud, with the data staying in the company’s systems and the model receiving only what is needed
Thousands of similar documents to classify every dayOn-premise, or a small model in the cloud: confidentiality decides
A company with data of every kindHybrid, with written rules on what goes where

The myths

  • “In the cloud, my data trains the model.” With the plans for organisations and the APIs of the main providers, as a rule no, by contract; it has to be read plan by plan, because in some cases (for example Mistral’s API) it depends on the plan chosen. With personal accounts, often yes.
  • “On-premise, I am covered on privacy.” You remove one risk, not the obligations.
  • “An open model is free.” The model, yes. The machine, the energy and whoever maintains it, no.
  • “On-premise I get the same quality.” It depends on the task: it has to be measured on your own work.
  • “Once you have chosen, there is no going back.” You can go back if the system was built so that the model can be replaced.

Questions to ask a supplier

  • Is my data used to train the models? Where is that written?
  • In which country does the processing take place, and where is the data stored?
  • How long are requests and responses kept, and who can read them?
  • Do you have a data processing agreement (DPA) that we can sign?
  • If you are proposing an on-premise solution: who updates it, how often, at what cost?
  • What happens if the machine stops?
  • What licence does the model you install have?
  • If tomorrow we wanted to change model or supplier, what would have to be redone?

How we work

At FT Support we choose case by case, and we start from the data. Archives and documents can stay on the company’s servers; the model receives only what is needed for that request. We mainly use models in the cloud through business APIs, with terms that exclude training, and the processing takes place in the Microsoft Azure cloud, in a European region. Where confidentiality or the lack of a network requires it, we design the system so that it can also work with an on-premise model: we assess costs and limits together, case by case.

In every project we keep the system independent of the model: instructions, data connections, checks and the log do not change if the model is replaced. This is what makes it possible to start in the cloud and, later on, to move on-premise only what needs it.

The choice between cloud and on-premise is one of the rules that fall to the people who run the company: we discuss it in the guide AI adoption in companies: top-down or bottom-up and, for the work on data, on the data and integrations page.

Sources

  1. ISTAT, Imprese e ICT, 2025 report (in Italian): use of cloud services by businesses
  2. OpenAI, Data controls in the OpenAI platform (data sent to the API, retention, regions)
  3. OpenAI, How your data is used to improve model performance (personal accounts)
  4. Anthropic, Is my data used for model training? (commercial products)
  5. Anthropic, How long do you store my organization's data?
  6. Anthropic, Data residency (regions available on the API)
  7. Anthropic, Updates to Consumer Terms and Privacy Policy (personal accounts)
  8. Microsoft, Data, privacy, and security for Foundry Models sold by Azure
  9. Google Cloud, Data residency (Gemini Enterprise Agent Platform)
  10. Google, Gemini Apps Privacy Hub (personal accounts)
  11. Amazon Web Services, Amazon Bedrock abuse detection (retention of requests and responses)
  12. Mistral AI, Where do you store my data or my Organization's data?
  13. Google, Gemma: memory required by model size and compression level
  14. OpenAI, model card for gpt-oss-120b (memory requirements and licence)
  15. Meta, Llama 4 Acceptable Use Policy (restriction for companies in the European Union)
  16. NVIDIA, GeForce RTX 50 series graphics cards: memory by model
  17. NVIDIA, RTX PRO 6000 Blackwell: memory and power consumption
  18. NVIDIA, DGX B200: memory and power of a data centre server
  19. AMD, Ryzen AI Halo: unified memory
  20. Stanford HAI, AI Index Report 2026, technical performance
  21. Regulation (EU) 2016/679 (GDPR), Articles 28 and 44-46
  22. European Data Protection Board, questions and answers on the Data Privacy Framework for European businesses (version 2.0)
  23. Court of Justice of the European Union, press release No. 106/25 on Case T-553/23 (Data Privacy Framework)
  24. EDPB and EDPS, legal assessment of the impact of the US CLOUD Act (2019)
  25. Regulation (EU) 2024/1689 (AI Act)
  26. European Commission, guidelines on the obligations of providers of general-purpose AI models
  27. Italy's Legislative Decree No. 138 of 4 September 2024 (NIS2), Article 24, on Normattiva (in Italian)
  28. Italy's National Cybersecurity Agency (ACN), NIS: procedures and basic specifications (in Italian)

Checked on 17 November 2026. Rules and deadlines change: always check the official source before you decide.

Frequently asked questions

The short answers

What is the difference between AI in the cloud and local AI, that is, on-premise?

A model in the cloud runs in a provider's data centres: the company uses it over the internet and pays by usage or by subscription. A local, or on-premise, model is an open-weight model installed on the company's computers or servers: the data does not leave the company, but you need hardware, maintenance and, usually, smaller models. Many companies use both, depending on how confidential the data is and how difficult the task is.

Can artificial intelligence be used without the internet?

Yes. An open-weight model installed on a company computer works even without a connection, because the processing takes place on the machine. It is the solution for departments with no network, isolated networks or data that must not leave the company. The limit is power: an office machine runs smaller models than those available in the cloud.

Does ChatGPT work offline? Can it be installed locally?

No. ChatGPT is a service that runs in OpenAI's data centres and requires a connection. What can be installed locally are open-weight models, including those published by OpenAI itself under the name gpt-oss, or those from Google, Mistral, Alibaba, Microsoft, IBM and others. They are different models from those of the online service, and generally less capable.

Is the data I type into ChatGPT, Claude or Gemini used to train the models?

It depends on the type of account. With personal accounts, usually yes, unless you switch the option off in the settings. With the plans for organisations and the APIs of the main providers, as a rule no, by contract, but it has to be read plan by plan: that is what the main providers' contract terms say. This is why the biggest risk for a company is the personal accounts used by its employees.

What hardware do you need to run AI locally?

Memory matters most: about 0.6 GB for every billion parameters of the model, plus the space for the context. A model with about 30 billion parameters runs on a workstation with 32-36 GB of memory; models with over 100 billion require machines with 96-128 GB, such as a mini PC with unified memory, a Mac Studio or a professional graphics card; the largest models require workstations with hundreds of GB of memory or data centre servers.

Is a local open-source model as good as the models in the cloud?

For summarising, classifying, extracting data from documents and answering questions on short texts, often yes. For multi-step reasoning, very long documents and agents that use tools, the most capable models are still those in the cloud. According to Stanford's AI Index 2026, the best open models are close to the best closed models, but they are too large for an office machine.

Is using an on-premise model enough to comply with the GDPR and the AI Act?

No. Keeping the data in-house removes the handover to a provider, but the other GDPR obligations remain: legal basis, privacy notices, security, retention periods. The AI Act obligations for those who use AI systems, starting with the measures for staff AI literacy, are the same in the cloud and on-premise.

Which is more cost-effective, the cloud or a server in the company?

It depends on how much you use it, and how. In the cloud you pay by usage or per user, with no investment; on-premise you buy the machine and the maintenance, and the cost does not change with use. With few, variable requests it is usage that counts; with high, steady volumes it is the fixed cost. More often, on-premise is chosen for confidentiality or to work without a network. We do not publish comparisons in figures: the assumptions change from company to company and the sums have to be done case by case.

Can you start in the cloud and move on-premise later?

Yes, provided the system is designed so that the model can be replaced: instructions, data connections and checks must not depend on a single provider. It is the most prudent route: you start in the cloud, measure what is really needed and move on-premise only what requires it.

Who looks after the maintenance of a model installed in the company?

Someone has to: update the model and the software that runs it, check that the machine can handle the load, keep the configurations, step in if it stops. In the cloud this work is done by the provider and is included in the price. On-premise it has to be entrusted to someone in-house or to a supplier, and budgeted for.

Do you have a process you would like to see working better?

Tell us about it in a first conversation, with no obligation: together we work out whether and where AI brings a measurable benefit, and which step to start with.