Architecture pattern: data and analytics

IoT platform architecture: connect devices, keep readings, act on them

An Internet of Things (IoT) platform connects sensors, machines or vehicles to the cloud, keeps their readings and lets you see and control them. Here is how the pieces fit, which service plays each part on AWS, Azure and Google Cloud, and what to decide first.

Quick answer

An IoT architecture connects devices over MQTT or HTTPS to a managed device gateway that gives each device an identity. Readings flow through a stream into a time-series store for charts and alerts, while an optional edge gateway filters data and works offline. Commands and firmware updates travel back to the devices.

Best forSensor networks, connected machines, fleet and asset tracking, and smart buildings
Less suited whenA handful of devices that can post readings to a simple web interface, where a full platform is too much
Start withTen real devices with their own certificates, sending readings to one dashboard, before you plan for thousands
Main services on AWSAWS IoT Greengrass, Amazon Managed Service for Apache Flink, Amazon Location Service, AWS IoT Core, Amazon Kinesis Data Streams, Amazon Timestream for InfluxDB, Amazon Quick Sight
Main services on AzureAzure IoT Operations, Azure Stream Analytics, Azure Maps, Azure IoT Hub, Azure Event Hubs, Azure Data Explorer, Power BI
Main services on Google CloudGoogle Distributed Cloud, Dataflow, Google Maps Platform, Pub/Sub with a partner IoT platform, Pub/Sub, Bigtable, Looker

The idea in plain words

A connected device is a small computer with a sensor, a radio and a battery or a power cable. It reads something, such as temperature, location or vibration, and sends it over Wi-Fi, a mobile network or a low-power radio network. In the cloud, a device gateway accepts those messages, knows which device sent each one and passes them on.

Behind the gateway, the design looks like a streaming pipeline built for time series: readings are streamed, checked against rules, stored in a database that is good at timestamps and shown on dashboards. Because devices sit in the field, the platform must also manage them: give each one a secret identity, send updates and commands, and notice when one goes quiet.

The edge matters when connections are poor or data is heavy. A small gateway near the devices can filter readings, run simple models and keep working offline. Plan for the whole life of the device, including security patches years from now.

The architecture

Boxes show the job each part does. Pick a cloud to see the service for each job.

IoT platform: architecture diagramTwo rows of boxes. The numbered steps below describe the same flow in words.filtered datawatches readingslocation on a mapcommands and updates1. Optional: the edge, live rules and maps (add as needed)Edge gatewayFilters data and worksofflineAWS IoT GreengrassAzure IoTOperationsGoogle DistributedCloudLive rulesSpots faults andtrends as they occurAmazon ManagedService for ApacheFlinkAzure StreamAnalyticsDataflowMapsShows where things areAmazon LocationServiceAzure MapsGoogle MapsPlatform2. From device to dashboardDevicesSensors, machines andvehiclesDevice connectionIdentity, securemessages and commandsAWS IoT CoreAzure IoT HubPub/Sub with apartner IoTplatformTelemetry streamHolds readings so manyreaders can use themAmazon KinesisData StreamsAzure Event HubsPub/SubTime-series storeReadings kept by timefor charts and trendsAmazon Timestreamfor InfluxDBAzure DataExplorerBigtableDashboardsLive views, reportsand alertsAmazon Quick SightPower BILookerIoT platform: architecture diagramBoxes from top to bottom. The numbered steps below describe the same flow in words.filtered datawatches readingslocation on a map1. Optional: the edge, live rulesand maps (add as needed)MapsShows where things areAmazon Location ServiceAzure MapsGoogle Maps PlatformLive rulesSpots faults and trends as theyoccurAmazon Managed Service forApache FlinkAzure Stream AnalyticsDataflowEdge gatewayFilters data and works offlineAWS IoT GreengrassAzure IoT OperationsGoogle Distributed Cloud2. From device to dashboardDevicesSensors, machines and vehiclesDevice connectionIdentity, secure messages andcommandsAWS IoT CoreAzure IoT HubPub/Sub with a partner IoTplatformTelemetry streamHolds readings so many readers canuse themAmazon Kinesis Data StreamsAzure Event HubsPub/SubTime-series storeReadings kept by time for chartsand trendsAmazon Timestream forInfluxDBAzure Data ExplorerBigtableDashboardsLive views, reports and alertsAmazon Quick SightPower BILooker↩ commands and updates, back to devices
Arrows show the direction of the flow, and dashed lines connect the rows. Each box names the main service for that job, taken from CloudBrum's service suggester.

Step by step

  1. Give each device an identityEvery device gets its own certificate or key at the factory or on first start, so the cloud can tell devices apart and block one that is lost or cloned. A shared password across devices is the weakest choice.
  2. The device connectsDevices send small messages over a light protocol such as MQTT, or over HTTPS, to a device gateway in the cloud. Devices on low-power radio networks reach the cloud through a local gateway.
  3. The edge filters (optional)A gateway near the devices can drop noise, combine readings, run a small model and store data while the link is down, then send the rest when the connection returns.
  4. Readings enter a streamThe device gateway routes each message by rule into a stream, so several readers, such as storage, alerts and analytics, can use the same readings.
  5. Rules watch live dataA stream processor or rule engine checks readings as they arrive, for example a motor running hot for five minutes, and raises an alert or starts an action.
  6. Readings are stored by timeA time-series store keeps readings with their timestamps and answers range queries fast. Detail is kept for recent weeks and averaged for older periods, with raw files archived cheaply.
  7. People see and controlDashboards show current values, maps and trends. Operators send commands or firmware updates back through the device gateway, and the platform records which devices applied them.

Services side by side

The main service for each job on each cloud, with a second option where one exists. Names checked on 23 October 2026.

JobAWSAzureGoogle Cloud
Run code and AI on edge devices
  • AWS IoT GreengrassRuns Lambda functions, containers and ML at the edge
  • Azure IoT OperationsRuns data and workloads at the edge with Azure Arc
  • Azure IoT Edge
  • Google Distributed CloudEdge and on-premises Google Cloud services
Real-time stream processing and analytics
  • Amazon Managed Service for Apache FlinkProcesses and analyses streams with Apache Flink
  • Azure Stream AnalyticsSQL-based real-time stream processing
  • Fabric Real-Time Intelligence
  • DataflowUnified stream and batch processing (Apache Beam)
Maps, geocoding and location
  • Amazon Location ServiceMaps, geocoding, routing and geofencing
  • Azure MapsMaps, search, routing and weather APIs
  • Google Maps PlatformMaps, places, routes and geocoding APIs
  • BigQuery GIS
Connect and manage IoT devices
  • AWS IoT CoreConnects devices over MQTT and HTTPS and routes data to AWS services
  • Azure IoT HubDevice connectivity, messaging and device twins
  • Device Provisioning Service
  • Pub/Sub with a partner IoT platformGoogle Cloud IoT Core was retired in 2023; use Pub/Sub for small fleets or a partner IoT platform
Real-time data streams (ingest events at scale)
  • Amazon Kinesis Data StreamsReal-time data streams at scale; Data Firehose delivers them to S3 or Redshift
  • Amazon MSK
  • Azure Event HubsBig-data event ingestion with a Kafka-compatible endpoint
  • Pub/SubGlobal messaging for event ingestion and streaming
Time-series database
  • Amazon Timestream for InfluxDBManaged InfluxDB for time-series data
  • Azure Data ExplorerFast analytics on telemetry, logs and time series; also in Microsoft Fabric as Eventhouse
  • BigtableA common choice for high-volume time-series data
  • BigQuery
Dashboards and business intelligence
  • Amazon Quick SightBI dashboards, part of Amazon Quick Suite (formerly QuickSight)
  • Power BIMicrosoft's BI and dashboard tool, built into Fabric
  • LookerGoverned BI and embedded analytics
  • Data Studio
Object storage (files, images, backups)Optional
  • Amazon S3Durable object storage for files, images, backups and data lakes
  • Azure Blob StorageObject storage for unstructured data, with hot, cool and archive tiers
  • Cloud StorageObject storage with Standard, Nearline, Coldline and Archive classes

Device connection works differently on each cloud. AWS and Azure provide a managed device service, while Google Cloud points to a messaging service combined with a partner IoT platform. See the service changes below.

Common variations

  • Devices go offline for hours or days. Buffer readings on the device or the edge gateway, send them with their original timestamps when the link returns, and design the store to accept late data.
  • Heavy data such as images or audio. Process it at the edge and send only results or short clips, because uploading raw streams from many devices is slow and costly.
  • Machines in a factory. Add an edge gateway that speaks the plant's protocols, such as OPC UA, and keep the machine network separate from the internet.
  • Vehicles or assets that move. Add location data and maps, store positions as time series and send alerts when something leaves an area.

Try this in the service suggester

Managed IoT service or your own message broker?

A managed IoT service handles device identity, messaging and fleet management. The alternative is to run an open-source message broker and build the device registry yourself, in the cloud or at the edge.

Managed IoT service

Good

  • Device identity, messaging and updates ready to use
  • Scales to very large fleets without broker servers
  • Built-in links to storage and analytics

Watch out

  • Pricing by message count can surprise at scale
  • Its own interfaces make moving harder
  • Service choices differ and change between clouds

Your own broker and registry

Good

  • Open standards and no per-message fee
  • Runs in any cloud, on-premises or at the edge
  • Full control of behaviour

Watch out

  • You run, patch and scale the brokers
  • Device identity and updates are yours to build
  • Security of the whole chain is on you
A common path. Use a managed service unless you have a firm reason not to. Keep devices on open protocols such as MQTT, so a later switch means changing endpoints, not firmware.

Keep in mind

  • Device security is the hardest part. Devices are physical, cheap and exposed. Use unique identities, signed firmware and encrypted links, close debug ports and plan how to revoke a stolen device. Standards such as ETSI EN 303 645 for consumer devices and IEC 62443 for industrial systems give useful checklists.
  • Updates must be possible. A device that cannot be updated becomes a liability. Build secure over-the-air updates with rollback into the first release, and decide how long you will support each model.
  • Rules for connected products. Connected products face growing rules, such as the EU Cyber Resilience Act, the UK's product security regime and national privacy laws. Check what applies in each market you sell to before you finalise the hardware.
  • Personal data in sensors. Cameras, wearables, vehicles and home devices reveal personal habits. Collect only what you need, tell users what you collect, and keep location and identity data apart where you can.
  • Plan for scale and cost. Cost grows with the number of devices, the messages each sends per day and how long you keep data. Send less often, batch readings, downsample old data and test with a realistic fleet before launch.
  • Connectivity differs by place. Mobile coverage, roaming costs, power and radio rules differ between countries and sites. Test devices where they will really run, and pick a network type that suits the battery life you need.
  • Time and clocks. Devices without a reliable clock send wrong timestamps. Sync time on connect, record both the device time and the arrival time, and handle readings that arrive late or out of order.
A starting point, not a design review. This is a general pattern. Your documents, users, rules and budget decide the real design. Confirm features, limits, pricing and regions on each provider's own pages. CloudBrum is independent of AWS, Microsoft and Google.

Words used on this page

Services in this pattern that have changed

Cloud providers rename and retire services often. These affect the parts above. Checked on 23 October 2026.

  • AWS Closed to new customers Amazon Timestream for LiveAnalytics closed to new customers on 20 June 2025. AWS recommends Amazon Timestream for InfluxDB. Details
  • AWS Renamed Amazon QuickSight is now Quick Sight inside Amazon Quick Suite (October 2025). Existing dashboards and APIs keep working. Details
  • Azure Closed to new customers New Azure IoT Central applications cannot be created from 23 September 2026; existing apps run until 20 September 2029. Microsoft points to IoT Hub, Device Provisioning Service and Microsoft Fabric. Details
  • Google Cloud Renamed Looker Studio is now called Data Studio. Details
  • Google Cloud Shut down Google Cloud IoT Core was discontinued on 16 August 2023. Google suggests an IoT platform from a partner, or Pub/Sub for small fleets. Details

Sources and further reading

Check details such as features, limits, prices and regions in the providers' own documentation.

Announcements about the changed services above

Frequently asked questions

What is an IoT platform?

An IoT platform is the cloud software that connects devices, receives their data, manages them and turns readings into charts, alerts and actions. Typical parts are a device gateway, a data stream, a time-series store, dashboards and tools for updates and fleet management.

What is MQTT and why is it used for IoT?

MQTT is a light messaging protocol where devices publish small messages to topics and others subscribe to them. It works well on slow or unreliable links and uses little battery, so it is one of the most common protocols between devices and an IoT gateway.

What is edge computing in IoT?

Edge computing processes data close to the devices, on a gateway or on the device itself, instead of in the cloud. It cuts delay and data costs, keeps working offline and lets you filter or protect data before it leaves the site.

Why use a time-series database for sensor data?

Sensor readings arrive in time order and are mostly queried by time range, such as the last hour or the same day last week. A time-series database stores and compresses them for that pattern, so range queries and rollups stay fast as data grows.

How many devices can a cloud IoT service handle?

Managed IoT services are built for very large fleets, into the millions of devices, but limits on connections, messages per second and message size apply. Check each provider's quotas against your numbers, and test with simulated devices before launch.

Want this reviewed for your case?

CloudBrum architects can adapt this pattern to your documents, users, rules and budget, and estimate the cost. Nothing is sent until you choose.

Building something else?

Describe it in plain words and get matching services on all three clouds.

Service suggester → Cloud consulting →