TiDB集群的安装,需要先安装一台中控机,然后通过中控机进行集群的安装及管理。
单机环境集群式将所有节点都安装在同一台服务器上。
在集群中,PD实例需要有3个,TiKV实例需要有3个,其余的实例可以只保留1个。
安装过程中,需要先安装中控机,然后通过中控机安装、管理集群。
通过命令下载并安装TiUP工具。
curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh | sh
安装过程为:
wux_labs@wux-labs-vm:~$ curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh | sh% Total % Received % Xferd Average Speed Time Time Time CurrentDload Upload Total Spent Left Speed
100 7088k 100 7088k 0 0 1483k 0 0:00:04 0:00:04 --:--:-- 1561k
WARN: adding root certificate via internet: https://tiup-mirrors.pingcap.com/root.json
You can revoke this by remove /home/wux_labs/.tiup/bin/7b8e153f2e2d0928.root.json
Successfully set mirror to https://tiup-mirrors.pingcap.com
Detected shell: bash
Shell profile: /home/wux_labs/.bashrc
/home/wux_labs/.bashrc has been modified to add tiup to PATH
open a new terminal or source /home/wux_labs/.bashrc to use it
Installed path: /home/wux_labs/.tiup/bin/tiup
===============================================
Have a try: tiup playground
===============================================
wux_labs@wux-labs-vm:~$
安装完成后,命令行中提示需要执行source /home/wux_labs/.bashrc
让环境变量生效,实际上这是因为在执行installs.sh的时候往这个文件中追加了:
export PATH=/home/wux_labs/.tiup/bin:$PATH
按提示执行命令让环境变量生效即可,此时,tiup命令被添加到环境变量PATH中。
执行以下命令安装cluster组件:
tiup cluster
安装过程为:
wux_labs@wux-labs-vm:~$ tiup cluster
tiup is checking updates for component cluster ...timeout(2s)!
The component `cluster` version is not installed; downloading from repository.
download https://tiup-mirrors.pingcap.com/cluster-v1.11.3-linux-amd64.tar.gz 8.44 MiB / 8.44 MiB 100.00% 7.00 MiB/s
Starting component `cluster`: /home/wux_labs/.tiup/components/cluster/v1.11.3/tiup-cluster
Deploy a TiDB cluster for productionUsage:tiup cluster [command]Available Commands:check Perform preflight checks for the cluster.deploy Deploy a cluster for productionstart Start a TiDB clusterstop Stop a TiDB clusterrestart Restart a TiDB clusterscale-in Scale in a TiDB clusterscale-out Scale out a TiDB clusterdestroy Destroy a specified clusterclean (EXPERIMENTAL) Cleanup a specified clusterupgrade Upgrade a specified TiDB clusterdisplay Display information of a TiDB clusterprune Destroy and remove instances that is in tombstone statelist List all clustersaudit Show audit log of cluster operationimport Import an exist TiDB cluster from TiDB-Ansibleedit-config Edit TiDB cluster configshow-config Show TiDB cluster configreload Reload a TiDB cluster's config and restart if neededpatch Replace the remote package with a specified package and restart the servicerename Rename the clusterenable Enable a TiDB cluster automatically at bootdisable Disable automatic enabling of TiDB clusters at bootreplay Replay previous operation and skip successed stepstemplate Print topology templatetls Enable/Disable TLS between TiDB componentsmeta backup/restore meta informationhelp Help about any commandcompletion Generate the autocompletion script for the specified shellFlags:-c, --concurrency int max number of parallel tasks allowed (default 5)--format string (EXPERIMENTAL) The format of output, available values are [default, json] (default "default")-h, --help help for tiup--ssh string (EXPERIMENTAL) The executor type: 'builtin', 'system', 'none'.--ssh-timeout uint Timeout in seconds to connect host via SSH, ignored for operations that don't need an SSH connection. (default 5)-v, --version version for tiup--wait-timeout uint Timeout in seconds to wait for an operation to complete, ignored for operations that don't fit. (default 120)-y, --yes Skip all confirmations and assumes 'yes'Use "tiup cluster help [command]" for more information about a command.
wux_labs@wux-labs-vm:~$
这样TiUP cluster组件就安装完成。
为了防止配置错误、提高配置效率,可以通过命令生成拓扑文件模板,然后基于模板修改拓扑配置。命令如下:
tiup cluster template > topology.yaml
由于是在一台服务器上部署多个实例,所以PD、TiKV的多个实例之间需要用不同的端口来进行区分,最终修改后的拓扑文件为:
global:user: "tidb"ssh_port: 22deploy_dir: "/tidb-deploy"data_dir: "/tidb-data"arch: "amd64"monitored:node_exporter_port: 9100blackbox_exporter_port: 9115server_configs:tidb:log.slow-threshold: 300tikv:readpool.storage.use-unified-pool: falsereadpool.coprocessor.use-unified-pool: truepd:replication.enable-placement-rules: truereplication.location-labels: ["host"]pd_servers:- host: wux-labs-vmclient_port: 23791peer_port: 23801deploy_dir: "/tidb-deploy/pd-23791"data_dir: "/tidb-data/pd-23791"log_dir: "/tidb-deploy/pd-23791/log"config:server.labels: { host: "logic-host-1" }- host: wux-labs-vmclient_port: 23792peer_port: 23802deploy_dir: "/tidb-deploy/pd-23792"data_dir: "/tidb-data/pd-23792"log_dir: "/tidb-deploy/pd-23792/log"- host: wux-labs-vmclient_port: 23793peer_port: 23803deploy_dir: "/tidb-deploy/pd-23793"data_dir: "/tidb-data/pd-23793"log_dir: "/tidb-deploy/pd-23793/log"tidb_servers:- host: wux-labs-vmtikv_servers:- host: wux-labs-vmport: 20161status_port: 20181deploy_dir: "/tidb-deploy/tikv-20161"data_dir: "/tidb-data/tikv-20161"log_dir: "/tidb-deploy/tikv-20161/log"config:server.labels: { host: "logic-host-1" }- host: wux-labs-vmport: 20162status_port: 20182deploy_dir: "/tidb-deploy/tikv-20162"data_dir: "/tidb-data/tikv-20162"log_dir: "/tidb-deploy/tikv-20162/log"config:server.labels: { host: "logic-host-2" }- host: wux-labs-vmport: 20163status_port: 20183deploy_dir: "/tidb-deploy/tikv-20163"data_dir: "/tidb-data/tikv-20163"log_dir: "/tidb-deploy/tikv-20163/log"config:server.labels: { host: "logic-host-3" }tiflash_servers:- host: wux-labs-vmmonitoring_servers:- host: wux-labs-vmgrafana_servers:- host: wux-labs-vmalertmanager_servers:- host: wux-labs-vm
由于是通过中控机安装、管理集群,虽然是单机环境的集群,也需要配置一下SSH免密登录。
wux_labs@wux-labs-vm:~$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/wux_labs/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/wux_labs/.ssh/id_rsa
Your public key has been saved in /home/wux_labs/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:sKT4o0ISwCqLtL0cVm4yFTFQS19FJbp3FsTyXfA/Gmg wux_labs@wux-labs-vm
The key's randomart image is:
+---[RSA 3072]----+
|. .o=. .o+oo.. |
|.. ..+ . ..o. ..|
|o .+. . o.. o|
|+. . = o . .....|
|+o+ = . S. E + ..|
|+o B o o o o .|
|o o O . |
|. + . |
| .. |
+----[SHA256]-----+
wux_labs@wux-labs-vm:~$
在安装集群之前,为了确保集群安装成功,需要先检查一下服务器是否满足安装集群的要求。执行以下命令进行检查:
tiup cluster check ./topology.yaml
检查过程为:
wux_labs@wux-labs-vm:~$ tiup cluster check ./topology.yaml
tiup is checking updates for component cluster ...
Starting component `cluster`: /home/wux_labs/.tiup/components/cluster/v1.11.3/tiup-cluster check ./topology.yaml+ Detect CPU Arch Name- Detecting node wux-labs-vm Arch info ... Done+ Detect CPU OS Name- Detecting node wux-labs-vm OS info ... Done
+ Download necessary tools- Downloading check tools for linux/amd64 ... Done
+ Collect basic system information
+ Collect basic system information- Getting system info of wux-labs-vm:22 ... Done
+ Check time zone- Checking node wux-labs-vm ... Done
+ Check system requirements
+ Check system requirements
+ Check system requirements
+ Check system requirements- Checking node wux-labs-vm ... Done- Checking node wux-labs-vm ... Done- Checking node wux-labs-vm ... Done- Checking node wux-labs-vm ... Done- Checking node wux-labs-vm ... Done- Checking node wux-labs-vm ... Done- Checking node wux-labs-vm ... Done- Checking node wux-labs-vm ... Done- Checking node wux-labs-vm ... Done- Checking node wux-labs-vm ... Done- Checking node wux-labs-vm ... Done- Checking node wux-labs-vm ... Done
+ Cleanup check files- Cleanup check files on wux-labs-vm:22 ... Done
Node Check Result Message
---- ----- ------ -------
wux-labs-vm sysctl Fail net.ipv4.tcp_syncookies = 1, should be 0
wux-labs-vm sysctl Fail vm.swappiness = 60, should be 0
wux-labs-vm sysctl Fail net.core.somaxconn = 4096, should be greater than 32768
wux-labs-vm thp Fail THP is enabled, please disable it for best performance
wux-labs-vm command Fail numactl not usable, bash: numactl: command not found
wux-labs-vm os-version Warn OS is Ubuntu 20.04.5 LTS 20.04.5 (ubuntu support is not fully tested, be careful)
wux-labs-vm cpu-cores Pass number of CPU cores / threads: 2
wux-labs-vm memory Pass memory size is 8192MB
wux-labs-vm selinux Pass SELinux is disabled
wux-labs-vm service Pass service firewalld not found, ignore
wux-labs-vm cpu-governor Warn Unable to determine current CPU frequency governor policy
wux-labs-vm network Pass network speed of enP58751s1 is 50000MB
wux-labs-vm network Pass network speed of eth0 is 50000MB
wux-labs-vm limits Fail soft limit of 'nofile' for user 'tidb' is not set or too low
wux-labs-vm limits Fail hard limit of 'nofile' for user 'tidb' is not set or too low
wux-labs-vm limits Fail soft limit of 'stack' for user 'tidb' is not set or too low
wux_labs@wux-labs-vm:~$
可以看到,检查结果中有失败的项目Fail
的。
这里可以手工修复不满足的项,也可以通过以下命令在检查的时候自动修复不满足的。
tiup cluster check ./topology.yaml --apply
命令执行以后,会重复刚才的检查动作,并且在检查完后会多出一步修复的动作,修复不满足的项。
修复完成后再次执行检查,确保配置项都是满足要求的。
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-oBTP8oCO-1676959836469)(https://gitee.com/wux-labs/Blogs/raw/master/images/image-20230215233846044.png)]
这里还差一个numactl软件需要手动安装,在Ubuntu 20.04系统中,执行命令安装软件:
sudo apt-get install numactl
软件安装完成后,再次检查一下,最终检查结果为全部通过。
由于我们指定了集群部署的目录是
deploy_dir: "/tidb-deploy"data_dir: "/tidb-data"
但是我们当前使用的用户并非root用户,所以需要先手动创建一下安装目录。
sudo mkdir /tidb-deploy /tidb-data
sudo chmod 777 /tidb-data /tidb-deploy
所有检查项都通过以后,通过deploy部署命令部署TiDB集群,其中cluster1是部署的集群的名字。
tiup cluster deploy cluster1 v6.1.0 ./topology.yaml
在等待确认的地方输入y,确认继续安装。
整个安装过程为:
wux_labs@wux-labs-vm:~$ tiup cluster deploy cluster1 v6.1.0 ./topology.yaml
tiup is checking updates for component cluster ...
Starting component `cluster`: /home/wux_labs/.tiup/components/cluster/v1.11.3/tiup-cluster deploy cluster1 v6.1.0 ./topology.yaml+ Detect CPU Arch Name- Detecting node wux-labs-vm Arch info ... Done+ Detect CPU OS Name- Detecting node wux-labs-vm OS info ... Done
Please confirm your topology:
Cluster type: tidb
Cluster name: cluster1
Cluster version: v6.1.0
Role Host Ports OS/Arch Directories
---- ---- ----- ------- -----------
pd wux-labs-vm 23791/23801 linux/x86_64 /tidb-deploy/pd-23791,/tidb-data/pd-23791
pd wux-labs-vm 23792/23802 linux/x86_64 /tidb-deploy/pd-23792,/tidb-data/pd-23792
pd wux-labs-vm 23793/23803 linux/x86_64 /tidb-deploy/pd-23793,/tidb-data/pd-23793
tikv wux-labs-vm 20161/20181 linux/x86_64 /tidb-deploy/tikv-20161,/tidb-data/tikv-20161
tikv wux-labs-vm 20162/20182 linux/x86_64 /tidb-deploy/tikv-20162,/tidb-data/tikv-20162
tikv wux-labs-vm 20163/20183 linux/x86_64 /tidb-deploy/tikv-20163,/tidb-data/tikv-20163
tidb wux-labs-vm 4000/10080 linux/x86_64 /tidb-deploy/tidb-4000
tiflash wux-labs-vm 9000/8123/3930/20170/20292/8234 linux/x86_64 /tidb-deploy/tiflash-9000,/tidb-data/tiflash-9000
prometheus wux-labs-vm 9090/12020 linux/x86_64 /tidb-deploy/prometheus-9090,/tidb-data/prometheus-9090
grafana wux-labs-vm 3000 linux/x86_64 /tidb-deploy/grafana-3000
alertmanager wux-labs-vm 9093/9094 linux/x86_64 /tidb-deploy/alertmanager-9093,/tidb-data/alertmanager-9093
Attention:1. If the topology is not what you expected, check your yaml file.2. Please confirm there is no port/directory conflicts in same host.
Do you want to continue? [y/N]: (default=N) y
+ Generate SSH keys ... Done
+ Download TiDB components- Download pd:v6.1.0 (linux/amd64) ... Done- Download tikv:v6.1.0 (linux/amd64) ... Done- Download tidb:v6.1.0 (linux/amd64) ... Done- Download tiflash:v6.1.0 (linux/amd64) ... Done- Download prometheus:v6.1.0 (linux/amd64) ... Done- Download grafana:v6.1.0 (linux/amd64) ... Done- Download alertmanager: (linux/amd64) ... Done- Download node_exporter: (linux/amd64) ... Done- Download blackbox_exporter: (linux/amd64) ... Done
+ Initialize target host environments- Prepare wux-labs-vm:22 ... Done
+ Deploy TiDB instance- Copy pd -> wux-labs-vm ... Done- Copy pd -> wux-labs-vm ... Done- Copy pd -> wux-labs-vm ... Done- Copy tikv -> wux-labs-vm ... Done- Copy tikv -> wux-labs-vm ... Done- Copy tikv -> wux-labs-vm ... Done- Copy tidb -> wux-labs-vm ... Done- Copy tiflash -> wux-labs-vm ... Done- Copy prometheus -> wux-labs-vm ... Done- Copy grafana -> wux-labs-vm ... Done- Copy alertmanager -> wux-labs-vm ... Done- Deploy node_exporter -> wux-labs-vm ... Done- Deploy blackbox_exporter -> wux-labs-vm ... Done
+ Copy certificate to remote host
+ Init instance configs- Generate config pd -> wux-labs-vm:23791 ... Done- Generate config pd -> wux-labs-vm:23792 ... Done- Generate config pd -> wux-labs-vm:23793 ... Done- Generate config tikv -> wux-labs-vm:20161 ... Done- Generate config tikv -> wux-labs-vm:20162 ... Done- Generate config tikv -> wux-labs-vm:20163 ... Done- Generate config tidb -> wux-labs-vm:4000 ... Done- Generate config tiflash -> wux-labs-vm:9000 ... Done- Generate config prometheus -> wux-labs-vm:9090 ... Done- Generate config grafana -> wux-labs-vm:3000 ... Done- Generate config alertmanager -> wux-labs-vm:9093 ... Done
+ Init monitor configs- Generate config node_exporter -> wux-labs-vm ... Done- Generate config blackbox_exporter -> wux-labs-vm ... Done
Enabling component pdEnabling instance wux-labs-vm:23793Enabling instance wux-labs-vm:23792Enabling instance wux-labs-vm:23791Enable instance wux-labs-vm:23791 successEnable instance wux-labs-vm:23792 successEnable instance wux-labs-vm:23793 success
Enabling component tikvEnabling instance wux-labs-vm:20163Enabling instance wux-labs-vm:20161Enabling instance wux-labs-vm:20162Enable instance wux-labs-vm:20163 successEnable instance wux-labs-vm:20162 successEnable instance wux-labs-vm:20161 success
Enabling component tidbEnabling instance wux-labs-vm:4000Enable instance wux-labs-vm:4000 success
Enabling component tiflashEnabling instance wux-labs-vm:9000Enable instance wux-labs-vm:9000 success
Enabling component prometheusEnabling instance wux-labs-vm:9090Enable instance wux-labs-vm:9090 success
Enabling component grafanaEnabling instance wux-labs-vm:3000Enable instance wux-labs-vm:3000 success
Enabling component alertmanagerEnabling instance wux-labs-vm:9093Enable instance wux-labs-vm:9093 success
Enabling component node_exporterEnabling instance wux-labs-vmEnable wux-labs-vm success
Enabling component blackbox_exporterEnabling instance wux-labs-vmEnable wux-labs-vm success
Cluster `cluster1` deployed successfully, you can start it with command: `tiup cluster start cluster1 --init`
wux_labs@wux-labs-vm:~$
至此,集群就算是安装完成了,接下来就可以启动集群了。
部署完成后,可以通过命令查看已安装的集群的信息。
tiup cluster list
tiup cluster display cluster1
可以看到,当前集群中有11个实例节点,但是都没有启动。
按照提示可以启动集群,其中--init
表示安全启动,启动后会给数据库的root用户生成一个密码。
tiup cluster start cluster1 --init
启动过程为:
wux_labs@wux-labs-vm:~$ tiup cluster start cluster1 --init
tiup is checking updates for component cluster ...
Starting component `cluster`: /home/wux_labs/.tiup/components/cluster/v1.11.3/tiup-cluster start cluster1 --init
Starting cluster cluster1...
+ [ Serial ] - SSHKeySet: privateKey=/home/wux_labs/.tiup/storage/cluster/clusters/cluster1/ssh/id_rsa, publicKey=/home/wux_labs/.tiup/storage/cluster/clusters/cluster1/ssh/id_rsa.pub
+ [Parallel] - UserSSH: user=tidb, host=wux-labs-vm
+ [Parallel] - UserSSH: user=tidb, host=wux-labs-vm
+ [Parallel] - UserSSH: user=tidb, host=wux-labs-vm
+ [Parallel] - UserSSH: user=tidb, host=wux-labs-vm
+ [Parallel] - UserSSH: user=tidb, host=wux-labs-vm
+ [Parallel] - UserSSH: user=tidb, host=wux-labs-vm
+ [Parallel] - UserSSH: user=tidb, host=wux-labs-vm
+ [Parallel] - UserSSH: user=tidb, host=wux-labs-vm
+ [Parallel] - UserSSH: user=tidb, host=wux-labs-vm
+ [Parallel] - UserSSH: user=tidb, host=wux-labs-vm
+ [Parallel] - UserSSH: user=tidb, host=wux-labs-vm
+ [ Serial ] - StartCluster
Starting component pdStarting instance wux-labs-vm:23793Starting instance wux-labs-vm:23791Starting instance wux-labs-vm:23792Start instance wux-labs-vm:23792 successStart instance wux-labs-vm:23791 successStart instance wux-labs-vm:23793 success
Starting component tikvStarting instance wux-labs-vm:20163Starting instance wux-labs-vm:20161Starting instance wux-labs-vm:20162Start instance wux-labs-vm:20162 successStart instance wux-labs-vm:20163 successStart instance wux-labs-vm:20161 success
Starting component tidbStarting instance wux-labs-vm:4000Start instance wux-labs-vm:4000 success
Starting component tiflashStarting instance wux-labs-vm:9000Start instance wux-labs-vm:9000 success
Starting component prometheusStarting instance wux-labs-vm:9090Start instance wux-labs-vm:9090 success
Starting component grafanaStarting instance wux-labs-vm:3000Start instance wux-labs-vm:3000 success
Starting component alertmanagerStarting instance wux-labs-vm:9093Start instance wux-labs-vm:9093 success
Starting component node_exporterStarting instance wux-labs-vmStart wux-labs-vm success
Starting component blackbox_exporterStarting instance wux-labs-vmStart wux-labs-vm success
+ [ Serial ] - UpdateTopology: cluster=cluster1
Started cluster `cluster1` successfully
The root password of TiDB database has been changed.
The new password is: '@2XKr^+9&nNZ3U07q6'.
Copy and record it to somewhere safe, it is only displayed once, and will not be stored.
The generated password can NOT be get and shown again.
wux_labs@wux-labs-vm:~$
从这个日志中可以看出,TiDB的启动顺序为:
启动完成后,再次查看一下集群的状态。
此时,TiDB的实例都启动了。
根据上面的结果,TiDB的Dashboard的地址为:http://wux-labs-vm:23792/dashboard,直接在浏览器中打开即可。
输入用户名和密码登录后可打开Dashboard界面。
这里面可以实现TiDB数据库集群的监控。
根据上面的结果,TiDB的Grafana的地址为:http://wux-labs-vm:3000,直接在浏览器中打开即可。
输入用户名和密码可以进入到监控界面。
除了上面几种,还可以直接查看Prometheus的监控数据,地址是http://wux-labs-vm:9090/,直接浏览器打开即可。
TiDB是分布式K-V数据库,对集群节点要求比较多,如果每个实例都分别在一台服务器上,则至少需要十几台服务器。单机环境仅作为模拟分布式集群用,不能用于生产环境。