1.1 Billion Taxi Rides with BrytlytDB & 2 GPU-Powered p2.16xlarge EC2 Instances

| |===============================+======================+======================| | 0 Tesla K80 Off | 0000:00:0F.0 Off | 0 | | N/A 61C P0 60W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 1 Tesla K80 Off | 0000:00:10.0 Off | 0 | | N/A 47C P0 69W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 2 Tesla K80 Off | 0000:00:11.0 Off | 0 | | N/A 66C P0 59W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 3 Tesla K80 Off | 0000:00:12.0 Off | 0 | | N/A 54C P0 72W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 4 Tesla K80 Off | 0000:00:13.0 Off | 0 | | N/A 61C P0 59W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 5 Tesla K80 Off | 0000:00:14.0 Off | 0 | | N/A 49C P0 70W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 6 Tesla K80 Off | 0000:00:15.0 Off | 0 | | N/A 66C P0 58W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 7 Tesla K80 Off | 0000:00:16.0 Off | 0 | | N/A 51C P0 70W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 8 Tesla K80 Off | 0000:00:17.0 Off | 0 | | N/A 62C P0 64W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 9 Tesla K80 Off | 0000:00:18.0 Off | 0 | | N/A 49C P0 72W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 10 Tesla K80 Off | 0000:00:19.0 Off | 0 | | N/A 63C P0 59W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 11 Tesla K80 Off | 0000:00:1A.0 Off | 0 | | N/A 52C P0 70W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 12 Tesla K80 Off | 0000:00:1B.0 Off | 0 | | N/A 65C P0 60W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 13 Tesla K80 Off | 0000:00:1C.0 Off | 0 | | N/A 52C P0 73W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 14 Tesla K80 Off | 0000:00:1D.0 Off | 0 | | N/A 64C P0 60W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ | 15 Tesla K80 Off | 0000:00:1E.0 Off | 0 | | N/A 52C P0 71W / 149W | 76MiB / 11439MiB | 0% Default | +——————————-+———————-+———————-+ The first machine has an IP address of 52.214.237.134 and has three roles: Global Transaction Manager, Coordinator and host to the first 16 data nodes..The second machine has an IP address of 34.250.232.38 and has two roles: it is also a coordinator and hosts the second set of 16 data nodes..TCP port 5432 is open between the two machines for communicating via PostgreSQLs wire protocol, TCP port 7777 is open for global transaction manager communication and ports 20,000 through to 20,031 are open for data node communication..In addition to the default 20 GB EBS volumes on each EC2 instance there are six 500 GB General Purpose SSDs attached which offer a baseline of 100 IOPS each and can burst to 3,000 IOPS if need be..These drives are setup in a RAID 0 configuration on each instance..Before setting up the RAID array Ill install a few dependencies..This was run on both EC2 instances..$ sudo apt update $ sudo apt install mdadm wget xfsprogs Below are the commands used to setup the RAID array on each instance..$ sudo mdadm –create –verbose –force /dev/md0 –level=0 /dev/xvdb –raid-devices=6 /dev/xvdc /dev/xvdd /dev/xvde /dev/xvdf /dev/xvdg $ sudo mkfs.ext4 /dev/md0 $ sudo mkdir /raidArray $ sudo mount /dev/md0 /raidArray $ sudo chown ubuntu /raidArray Heres what the RAID layout looked like after it was setup..$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 20G 0 disk └─xvda1 202:1 0 20G 0 part / xvdb 202:16 0 500G 0 disk └─md0 9:0 0 3T 0 raid0 /raidArray xvdc 202:32 0 500G 0 disk └─md0 9:0 0 3T 0 raid0 /raidArray xvdd 202:48 0 500G 0 disk └─md0 9:0 0 3T 0 raid0 /raidArray xvde 202:64 0 500G 0 disk └─md0 9:0 0 3T 0 raid0 /raidArray xvdf 202:80 0 500G 0 disk └─md0 9:0 0 3T 0 raid0 /raidArray xvdg 202:96 0 500G 0 disk └─md0 9:0 0 3T 0 raid0 /raidArray This RAID 0 setup offers a partition with a capacity of 3.2 TB on each instance: $ df -H Filesystem Size Used Avail Use% Mounted on udev 387G 0 387G 0% /dev tmpfs 78G 9.2M 78G 1% /run /dev/xvda1 21G 7.2G 14G 35% / tmpfs 387G 0 387G 0% /dev/shm tmpfs 5.3M 0 5.3M 0% /run/lock tmpfs 387G 0 387G 0% /sys/fs/cgroup tmpfs 78G 0 78G 0% /run/user/1000 /dev/md0 3.2T 57G 3.0T 2% /raidArray BrytlytDB Up & Running Ive run the following to download BrytlytDBs install script..BrytlytDB is commercial software so I cannot divulge the URL I pulled this from at this time..The following was run on both EC2 instances.. More details

Leave a Reply