system

Presto-engine Privileges Control – Architect

January 20, 2022 Architect, bigdata, linux, system No comments

We use presto as the main data query engine to provide data query services for business parties. In our architecture, presto as a core capability requires access to data query permission control, including permission control at the table and column levels. We designed The following scheme to achieve this requirement

(more…)

[Upgrade] MySQL database upgrade to 8.x version tips

December 6, 2021 MYSQL, software, system No comments

Recently we upgrade our database cluster to version 8.x .

Read this PDF to get some  experience    MySQL_8.x_upgrade

[Archiving] Antdeck Platform for MySQL

March 5, 2020 Architect, backup, MYSQL, rdbms, storage, system No comments

We use Antdeck paltform to do mysql data archiving jobs. This platform was developed by ruzuojun . He is also the author of  Lepus 

Read this pdf  Antdeck , we use TiDB as secondary storage .

[Hardware] Redis with Intel AEP Test

November 12, 2019 Architect, hardware, redis, storage, system No comments

Based on Netease Test

Read this PDF :AEP_REDIS_TEST

 

Reference:

从Intel Optane SSD和DIMM谈全闪存性能优化
Intel Data-Centric Innovation Day的启示(2)-Intel AEP
Optane DIMM写速度比SSD慢?内存接口可能并非完美
Intel Optane DC持久化内存性能测试、数据库应用优化

Donkey System

January 18, 2018 Architect, Architecture, linux, MYSQL, software, system No comments

Donkey system is a fully automatic MySQL database change system.
It gives a great help both to the release of the business and the company’s automated operation and maintenance.

Donkey.pptx
Donkey_intro.pdf

JKDB PLATFORM

August 21, 2017 system No comments

Introduction of JKDB. JK.CN‘s data management platform.

A platform for database /cache system management
A platform for resource control management
A platform for all of data management

Download this PPT JKDB_PLATFORM

[JKDB] BACKUP AND RESTORE

May 11, 2017 Architect, backup, hardware, IDC, maintain, MYSQL, rdbms, replication, system No comments

JKDB PLATFORM – our company’s data management platform

In this post, we introduce backup-restore module,focus on automatic backup and restore in a huge MySQL database environment.

Read this PDF: JKDB:BACKUP-RESTORE

Facebook WDT Test

March 17, 2017 Architect, backup, hardware, IDC, network, system No comments

We test Facebook WDT tool:https://github.com/facebook/wdt

Read this PDD: WDT-TEST

Log-Center for OPS

March 10, 2017 Architect, system No comments

We build a strong log-collection system to analyze total logs of our operation teams (including system ,database, web service ,load balance ,audit, cache system , hardware, iDrac logs). All of these logs are used to build a high streaming log system to find a particular performance point of our apps .We also analyze these logs in time to get a notification of attack or even a risk of event.

Download :log-center-arch.pdf

Donkey – system for MySQL automatic maintenance

February 17, 2017 Architect, Architecture, maintain, MYSQL, system No comments

We build this system based on Inception (Qunar)。

All of MySQL DDL/DML operations are based on Donkey system, with strict approval process to keep online system strong and available.

Open this file by Mockplus Donkey-system