Introduction to Altinity.Cloud — Platform for Real-Time Data | Managed Cloud for ClickHouse®

Introduction

The speakers introduce themselves and give a brief overview of the webinar.

• Robert Hodges and Alexander Zaitsev are hosting the webinar on alterity.cloud, a platform for real-time analytics on ClickHouse.

• The audience is given a couple of minutes to join before the presentation begins.

Sound Check

The speakers do a quick sound check before starting the presentation.

• Alexander confirms that his sound is on and video is fully functional.

Introduction to Altinity.Cloud

The speakers provide an introduction to Altinity.Cloud, their platform for real-time analytics on ClickHouse.

• Altinity.Cloud is a SaaS platform that operates in the cloud with baked-in Enterprise DBA support.

• It's optimized for real-time analytics, particularly systems designed to operate at scale across many tenants.

• It supports practically any version and feature of ClickHouse, including experimental ones.

• It runs anywhere - in any Amazon or GCP region or you can provide your own Kubernetes cluster and they will manage it there.

Quick Introductions

The speakers introduce themselves and their roles at Altinity.

• Robert Hodges is CEO of Altinity with 40 years of experience working with databases.

• Alexander Zaitsev has 35 years of experience working with databases and is also the designer of the system being presented.

• They assume that the audience consists of application developers looking to build real-time analytics to solve business problems without spending too much time running infrastructure.

About Altinity

The speakers provide more information about Altinity as a company.

• They are an enterprise provider for ClickHouse with enterprise support and cloud services that run ClickHouse.

• They are authors of a number of open-source packages, including the Kubernetes operator for ClickHouse.

• The Kubernetes operator for ClickHouse was the first operator ever written for a data warehouse in Kubernetes and is one of the most popular database operators on GitHub.

Introduction to Altinity.Cloud (Continued)

The speakers continue their introduction to Altinity.Cloud.

• Altinity.Cloud is optimized for real-time analytics with a special architecture focused on high-performance storage and compute.

• It has a cost model that is well-suited for businesses that run these applications.

• It supports practically any version and feature of ClickHouse, including experimental ones.

• It runs anywhere - in any Amazon or GCP region or you can provide your own Kubernetes cluster and they will manage it there.

Enterprise DBA Support

The speakers discuss the enterprise DBA support provided by Altinity.Cloud.

• Altinity.Cloud has highly expert support teams who have run large clusters and worked with ClickHouse for many years.

• Their role is to help users come up to speed quickly on ClickHouse and cover them when unusual or unexpected things happen.

Introduction to ClickHouse

In this section, the speaker introduces ClickHouse and its features.

Features of ClickHouse

• ClickHouse is an open-source columnar database management system that allows for efficient storage and communication over a network.

• It has a whole ecosystem of software around it that is similarly licensed either Apache or some other permissive license.

• It can scale from a laptop running a few megabytes of data all the way up to clusters with 500 nodes and dealing with tens of petabytes.

Demo: Alternative.cloud

In this section, the speaker gives a demo on how to use alternative.cloud.

Starting a Cluster

• The speaker starts by showing the cluster page which is used as a dashboard when managing your ClickHouse clusters.

• He then proceeds to start a new cluster in real-time called "webinar 2".

• You can pick different versions out into stable builds or Community builds depending on your needs.

• Once you configure what version you use, you select all kinds of resources you want to throw at this ClickHouse cluster.

Configuring Cluster Settings

• You can configure the storage size, number of volumes, volume type, number of shards, and whether or not to enable replication.

• The endpoint is created inside the dma dma column in authentic load but if your authentical customer you will get the dedicated third level domain on student cloud and all your clusters will be using the fourth level level domain.

• Most customers actually run private access because we support the protocols binary and HTTP with some of our clients also using MySQL protocol to access ClickHouse.

• Uptime schedule is a feature that allows you to stop the cluster automatically when it's idle for example when you don't use a system.

Rescaling

• The speaker shows how to use the rescale feature which allows you to add replicas or change node size.

Introduction to Altinity.Cloud

In this section, Alexander and Robert introduce Altinity.Cloud and explain how it works. They discuss the benefits of using cloud storage over on-premises storage, and demonstrate how easy it is to add a replica to a cluster.

Benefits of Cloud Storage

• Customers can increase their storage capacity without limits.

• Adding a replica does not require downtime.

• It takes 3-5 minutes to start single nodes, but after 20 minutes, the cluster will be fully up and running.

Features of Altinity.Cloud

• Users can stop and resume clusters as needed.

• Supports any server settings, profile settings, or user settings.

• Built-in database explorer allows users to run queries, check schema, analyze workloads, data consistency or inconsistency.

• Users can configure alerts and access logs for monitoring data.

Features of Altinity.Cloud (Continued)

In this section, Robert provides more detail about some of the features in Altinity.Cloud. He explains where clusters live when they are set up and how environments work. He also discusses how users can run in any region with practically any cloud environment.

Where Clusters Live

• Clusters live inside an environment which is actually a VPC running on Amazon using eks or GCP.

• When a tenant is provisioned for the first time, we set up a VPC with associated networking and provision a kubernetes cluster inside it.

Running in Any Region

• Users can run in any region on our account or their own account.

• Altinity.Cloud supports practically any cloud environment, and if it's one we haven't used before, we may have to wait a few hours to get quota increases.

Introduction to Alternator.Cloud

This section introduces Alternator.Cloud and its three different environments. It also explains how the environment is set up, what services are included, and how it differs from other cloud providers.

What is Alternator.Cloud?

• Alternator.Cloud is a cloud service that provides managed ClickHouse clusters.

• It has three different environments: AWS, GCP, and Anywhere.

• The environment is set up as a Kubernetes cluster with secure networking links.

• Services included in the environment are ClickHouse, alternative operator, Grafana, Prometheus, Edge proxy, block storage allocation for services that use storage and object storage buckets for backups.

How does it differ from other cloud providers?

• Alternator.Cloud operates exactly like running ClickHouse at home.

• All software used in the stack is open source under Apache license or similar licenses.

• Users can run any version of ClickHouse and use virtually any feature including experimental ones.

• Applications connecting to an Alternator.Cloud cluster must support TLS and server name indications (SNI).

• Everything else that works on a laptop will work on Alternator.Cloud.

Developing Real-Time Analytics on Alternator.Cloud

This section discusses tips for developing real-time analytics on Alternator.Cloud. It covers starting a trial account, viewing clusters, launching clusters, stopped clusters vs running clusters and their costs.

Tips for Developing Real-Time Analytics

• Start by creating a trial account on Alternator.Cloud.

• View available clusters which include both stopped (no compute but still billed for block storage) and running clusters (compute active).

• Launch new clusters using the launch cluster button in the cluster view.

• Stopped clusters are useful for development since compute is the most expensive part.

ClickHouse Builds and Versions

This section discusses the different versions of ClickHouse and which ones are suitable for production use.

ClickHouse Builds

• ClickHouse produces monthly Community builds and every six months, a long-term support (LTS) build.

• The LTS build has one year of support while the monthly builds have a month of actual community support.

• The alternative stable builds created by Altinity are slower moving, based only on the LTS builds, and go through a certification period that lasts several months before being declared stable.

• For production systems, it is recommended to use alternative stable builds as they are more stable and avoid problems that pop up in community builds.

Connecting to a Node in a Cluster

This section explains how to connect to a specific node in a cluster using Kubernetes.

Accessing Cluster Information

• Kubernetes allows access to the cluster as a whole or explicit nodes.

• The cluster endpoint provides load balancing while explicit nodes allow connecting to specific nodes for loading data from Kafka or other applications.

• The cluster dashboard provides information about the node type, storage, number of nodes, and connection details for the cluster endpoint.

Connecting to a Node

• Each node has its own DNS name provided by Altinity.

• To connect to a specific node, click on the "nodes" link in the cluster dashboard.

Introduction to ClickHouse

In this section, the speaker introduces the Explore tab and how to create users.

Explore Tab

• The Explore tab is located on the dashboard and provides a simple query editor.

• It allows you to go to specific clusters and pick a node endpoint.

• You can also pick specific nodes for queries where you don't care which node you land on.

Creating Users

• To create users, go to the Configure tab and click Users.

• Fill in user details such as username, password, and restrict databases if necessary.

• Enable access management switch to allow users to issue SQL R back commands for role-based access control.

• It's better to create one login instead of many XML-based user definitions out on ClickHouse.

More on User Creation

In this section, the speaker explains how creating a user propagates instantly through SQL.

Using Created User

• After creating a user, it typically takes a minute or two for that user to propagate out because alternating.cloud updates the file system where each of the alternator always give me the ClickHouse nodes is running.

• Once properly loaded onto the node and ClickHouse notices your user then appears.

Analytics with ClickHouse

In this section, the speaker discusses useful tools available in ClickHouse's Explore tab for analytics.

Schema Tab

• The Schema tab is useful for looking at your schema.

• You can see data size, number of partitions being created, bytes per row compression ratios using this tab.

Workload Tab

• The Workload tab shows what queries are running and how long they're taking.

DBA Tools

• There are various DBA tools available in ClickHouse's Explore tab like detecting detached parts.

Open Source Analytics Software

• ClickHouse has an enormous ecosystem of open-source analytic software.

• Some commonly used tools include event streaming, extract transfer load, Apache Spark, and client libraries for just about every single language.

Operating Your Application

In this section, the speaker discusses how to operate your application using ClickHouse.

Day Two Operations

• Day two operations refer to operating your application after it's been deployed.

• The speaker will dig in a little bit more deeply into day two operations.

Restarting and Scaling a Cluster

This section covers how to restart and scale a cluster in Kubernetes.

Restarting a Cluster

• To restart the cluster, go to the actions dropdown and select "restart".

• If you don't hit the checkbox, it will do a rolling restart.

• Queries will finish before restarting.

• Large numbers of files can contribute to longer restart times.

Turning off Compute

• Use the stop cluster action to suspend compute.

• Resuming allows for changing node type.

• Uptime schedules can be set for automatic operation during specific days and times of week.

Rescaling Options

• Scaling horizontally increases shards and replicas.

• Scaling vertically changes node size or adds storage.

• Data size tends to grow over time, so extending storage is common.

• Changing VM size terminates old VM and provisions new one.

• Extending storage either extends individual volume on GCP or stacks volumes on Amazon.

ClickHouse Cloud Features

This section covers the features of ClickHouse Cloud, including vertical scaling, horizontal scaling replicas and shards, backups, and upgrade/downgrade.

Vertical Scaling

• The user can increase the allocated resources at any time.

Horizontal Scaling Replicas and Shards

• Replicas and shards are scaled automatically by the operator.

• When a new replica or shard is added, the schema is provisioned automatically.

• Replicated tables are defined on new replicas automatically.

• ClickHouse does not do automatic rebalancing across shards yet.

Backups

• Backups run automatically every 24 hours with seven days of backups by default.

• Users can restore one backup at any time or create one manually if needed.

Upgrade/Downgrade

• Upgrade/downgrade operations are completely automated in a round-robin fashion.

• Users can choose between alternative builds versus community builds when upgrading.

• It's important to test upgrades before implementing them on mission-critical systems.

ClickHouse Alternative Support

In this section, the speaker talks about the importance of seeking expert advice from alternative support when using ClickHouse. They also discuss the need to inform them before a major upgrade.

Seeking Expert Advice

• Alternative support provides expert advice for complex issues with ClickHouse.

• Major upgrades should be communicated beforehand to get help and advice in advance.

Monitoring

• Three types of monitoring panels are available: cluster metrics, query performance, and logs.

• Cluster metrics show inserts, merge activity, queries running, and system load.

• Query performance shows top-end queries.

• Logs are easily accessible by clicking on the view in Grafana button/link.

System Status

• System status can be accessed by finding your name and clicking on it.

• The system status page gives an overview of the general health of the system.

• It may not catch everything like transient failures but gives a good sense of what's going on with your system.

Security Features in ClickHouse Alternative Support

This section discusses security features built into ClickHouse alternative support. The speaker talks about encryption, IP whitelisting, VPC endpoints, and controlling access to data.

Protecting Applications and Data

• All traffic is TLS encrypted except for rare exceptions like MySQL protocol parts that require special requests.

• IP whitelisting can increase security by allowing only authorized IPs to connect.

• VPC endpoint setup allows private networking between VPCs over a private network that's not visible to anyone else.

Controlling Access to Data

• A feature on the cluster dashboard allows users to control access levels given to alternative support.

• Access levels range from no access to full access.

• Read-only access allows support to look into system tables but not run queries on databases containing user data.

Adding People to the Organization

This section explains how to add more people to the organization.

Adding People

• To add more people, go into the cluster dashboard and hit the share link.

• Type in emails and they'll get an automatically generated invite that will allow them to connect.

Enterprise Support

This section explains what Enterprise support is and how it works.

Unique Features of Alternator.Cloud

• Alternator.cloud has been doing support for ClickHouse since 2017.

• They have helped hundreds of customers ranging from tiny crypto startups all the way to Fortune 100 companies.

• ClickHouse is a complex piece of software that requires some knowledge about how it works inside as well as what are the latest PR's that have been added.

Types of Support Provided

• You can file cases, use slack or zendesk for support.

• The types of things that they can help you with range from questions on schema design to troubleshooting problems particularly in live systems.

• They can also help with upgrades, capacity planning, integrations with other systems, and random questions like how do I migrate from one system to another.

Getting the Most Out of Support

• Log production issues in zendesk so that both parties can see that it's getting closure.

• Contact them in advance if you're doing something big like an upgrade or scale-out operation that may require raising quotas because we allow everybody to run anywhere you can actually hit quota limits.

• Don't wait for problems so they can help you up front. Feel free to ask questions on anything.

Getting Started with Alternate.Cloud

In this section, the speaker provides information on how to get started with Alternate.Cloud.

Free Two-Week Trial

• To get started with Alternate.Cloud, click on the provided link.

• Fill out a form to kick off a free two-week trial.

• During the trial period, you will receive full support and can evaluate whether this is the right thing for you.

Q&A Session

In this section, the speaker opens up the floor for questions from attendees.

Questions and Thanks

• Attendees are encouraged to ask any questions they may have by putting them into the Q&A box or chat.

• The speaker thanks Alexander for his help on the presentation.

Conclusion and Farewell

In this section, the speaker concludes the presentation and thanks attendees for their participation.

Final Remarks

• The slides from the presentation will be published.

• A recording of the presentation will also be published for those who signed up.

• Attendees are thanked for joining and it is hoped that they found it helpful.

• The speaker looks forward to seeing attendees on Alternate.Cloud.