HarmonyOS鸿蒙Next中MTRdb-多线程可共享对象的ORM库

MTRdb-多线程可共享对象的ORM 库

前言

因为ArkTs的运行时特性，线程间的对象是隔离的。Harmony 4.1 开始，系统提供了@Sendable装饰器，用于修饰在多线程间共享的对象。但使用@Senable时，不可再混用其他装饰器，因此目前的 ORM 库，均不适用。基于此，我们对 RDB 的 API 进行了封装，在不使用装饰器的情况下，实现了ORM，生成对象可用于多线程间共享传递。可以用在具有复杂多线程交互的应用当中。

Demo 演示

演示demo仓库：https://gitee.com/geno/mtrdb_demo

实现

初始化配置

MTRdb.ets init 传入 config，MTRdb保存好 application 以及数据库配置，供后续创建、获取 rdbStore；

export class MTRdb {
  private static isInit: boolean = false
  private static _config?: MTRdbConfig
  // …… 省略 

  static init(config: MTRdbConfig) {
    if (MTRdb.isInit) {
      return
    }
    MTRdb._config = config
    if (config.enableLog !== undefined) {
      MTRdbLog.enableLog(config.enableLog, config.logLevel)
    }
    MTRdb.isInit = true
  }
}

获取 RdbStore

BaseTableDao.ets 实现了部分 DAO 层常用的接口；初始化时，会构建RdbCore，传入 BaseTableInterface（见本节尾部说明）； 其中getRdbStore会从RdbCore的getRdbStore方法获取 RdbStore;

export abstract class BaseTableDao<T extends BaseModel> {
  public baseDao?: RdbCore = undefined;
  public tableConfig?: BaseTableInterface = undefined;
  private uniqueColumnName: string[] = []

  /**
   * 构建 Model
   * @returns  返回 model 对象
   */
  abstract generateModelObj(): T

  /**
   * 保存数据库时，需要保存成 string时，在此处理
   * @param model
   */
  abstract saveParamsToString(model: T)

  /**
   * 数据库读取对象后，需要将 string 重新映射到 model，再次处理
   * @param model
   */
  abstract getParamsFromString(model: T)

  constructor(tableConfig: BaseTableInterface, callback: Function = () => {
  }) {
    this.tableConfig = tableConfig;
    this.baseDao = new RdbCore(tableConfig);
    this.uniqueColumnName = tableConfig.uniqueColumnName
    this.baseDao.getRdbStore(callback);
  }

  getRdbStore(callback: Function = (rdb?: relationalStore.RdbStore) => {
  }) {
    this.baseDao?.getRdbStore(callback);
  }
  
  //略
}

获取代码如下，会从第一步初始化配置的MTRdb中获取 context、rdbconfig ；以此获取 RdbStore；创建表，检查升级等

export class RdbCore {
  private rdbStore: relationalStore.RdbStore | null = null;
  private tableName: string;
  private sqlCreateTable: string;
  private columns: Array<string>;
  private migrationContainer = new Array<Migration>()
  /**
   * 升级配置
   */
  updateConfig?: BaseUpConfig
  constructor(tableConfig: BaseTableInterface) {
    this.tableName = tableConfig.tableName;
    this.sqlCreateTable = tableConfig.sqlCreate;
    this.columns = tableConfig.columnNames;
    this.updateConfig = tableConfig.updateConfig
  }

  getRdbStore(callback: Function = (rdb?: relationalStore.RdbStore) => {
  }) {
    //略
    if (this.rdbStore !== null) {
      MTRdbLog.i(TAG, 'The rdbStore exists.');
      callback(this.rdbStore);
      return
    }
    let context: Context | undefined = MTRdb.appContext;
    if (context) {
      relationalStore.getRdbStore(context, MTRdb.rdbConfig, (err, rdb) => {
        if (err) {
          MTRdbLog.e(TAG, `gerRdbStore() failed, err: ${err}`);
          return;
        }
        this.rdbStore = rdb;
        if (this.rdbStore.version === 0) { //第一次使用
          let predicates = new relationalStore.RdbPredicates(this.tableName).limitAs(1);
          let resultSet = this.rdbStore.querySync(predicates)
          if (resultSet.columnCount === 0) { // 创建数据表
            // 设置数据库的版本
            this.rdbStore.version = MTRdb.dbVersion
          }
        }
        this.rdbStore.executeSql(this.sqlCreateTable); //创建表
        this.checkDbMigration(this.rdbStore)
        MTRdbLog.d(TAG, 'getRdbStore() finished.');
        callback(this.rdbStore);
      });
    } else {
      MTRdbLog.e(TAG, `context is null`);
      callback(undefined)
      return;
    }
  }

  //略。。。
}

BaseTableInterface.ets 接口定义

需要实现表名、创建语句等属性

export interface BaseTableInterface {
  /**
   * 表名
   */
  tableName: string;
  /**
   * 表的创建语句
   */
  sqlCreate: string;
  /**
   * 表列名
   */
  columnNames: string[];
  /**
   * 主键列名
   */
  uniqueColumnName: string[];

  /**
   * 更新配置信息，可以配置也可以不配置
   */
  updateConfig?: BaseUpConfig
}

CRUD增删改查

在上步提到BaseTableDao.ets中，定义了通用的 CRUD 接口

增加、修改

增改时，只需传入对应的Model；内部会解析Model的属性，构建出 valueBucket（generateBucket）;之后调用 RdbCore 对应的方法，进行增、改

 async insert(model: T) {
   //略
    const valueBucket: relationalStore.ValuesBucket = this.generateBucket(model, model.getDbIgnoreObjProp());
    return await new Promise<boolean | number>((resolve, _) => {
      this.baseDao?.insertData(valueBucket, (result: boolean | number) => {
        resolve(result)
      })
    });
  }
  async update(model: T) {
    //略
    const valueBucket: relationalStore.ValuesBucket = this.generateBucket(model, model.getDbIgnoreObjProp());
    let predicates = new relationalStore.RdbPredicates(this.tableConfig.tableName);
    this.uniqueColumnName.forEach(column => {
      predicates.equalTo(column, model.getUniqueProp().get(column));
    })
    return await new Promise<boolean | number>((resolve, _) => {
      this.baseDao?.updateData(predicates, valueBucket, (result: boolean | number) => {
        resolve(result)
      });
    });
  }
// 其他略

其中generateBucket实现逻辑简单过程为：获取 model的属性，再根据 BaseTableInterface中实现的列名过滤，通过TSUtils获取 model 的属性值，填充到对应的ValuesBucket中

/**
 * 生成插入数据库的数据ValuesBucket
 * @param model
 * @returns
 */
protected generateBucket(
  model: T,
  ignoreProperties: string[],
  embeddedPrefix?: string
): relationalStore.ValuesBucket {
  if (this.tableConfig == undefined) {
    return {}
  }
  let rdbBucket: relationalStore.ValuesBucket = {};
  //获取 model的属性
  let propNames: string[] = Object.getOwnPropertyNames(model);
  //多层嵌套对象的属性，需要增加前缀（ "嵌套对象的命名" + "_"）对应上数据库字段
  let prefix = embeddedPrefix == undefined ? '' : `${embeddedPrefix}_`
  propNames.forEach((propName, _) => {
    if (this.tableConfig?.columnNames?.includes(prefix + propName)) {
      rdbBucket[prefix+propName] = TSUtils.getPropValue(model, propName)
    } else {
      let type = TSUtils.getType(model, propName)
      if (type == 'object' && !ignoreProperties.includes(prefix + propName)) {
        let subBucket =
          this.generateBucket(TSUtils.getPropValue(model, propName), ignoreProperties, prefix + propName)
        let propNames: string[] = Object.getOwnPropertyNames(subBucket);
        propNames.forEach((propName, _) => {
          if (this.tableConfig?.columnNames?.includes(propName)) {
            rdbBucket[propName] = TSUtils.getPropValue(subBucket, propName)
          }
        })
      }
    }
  })
  return rdbBucket;
}

TSutils.ts 使用ts 是为了规避 ets 的检测规则，实现对 model 属性的获取和赋值；

/**
 * ts 工具，用来规避 arkts 的一些检测
 */
export namespace TSUtils {
  export function getType(obj: any, propName: string): string {
    return typeof obj[propName]
  }

  export function getPropValue(obj: any, propName: string): any {
    return obj[propName]
  }

  export function setPropValue(obj: any, propName: string, propValue: any) {
    obj[propName] = propValue
  }
}

删除

删除时，同样只需要传入对应的 Model；内部通过 BaseTableinterface 的实现类，获取到对应的表名以及主键列名；再通过Model的对应的getUniqueProp接口（BaseModel接口定了Model需要实现的接口），获取对应的主键值

 async delete(model: T) {
    //略
    let predicates = new relationalStore.RdbPredicates(this.tableConfig.tableName);
    this.uniqueColumnName.forEach(column => {
      predicates.equalTo(column, model.getUniqueProp().get(column));
    })

    return await new Promise<boolean | number>((resolve, _) => {
      this.baseDao?.deleteData(predicates, (result: boolean | number) => {
        resolve(result)
      })
    });
  }

BaseModel.ets 接口定义；支持 Sendable

/**
 * 表model 基础接口，其他model需实现此接口，getUniqueProp返回表的主键值;
 */
export interface BaseModel extends lang.ISendable {
  //返回表的主键值
  getUniqueProp(): Map<string, ESObject>

  //返回需要忽略的二级对象的属性名(忽略就会解析)
  getDbIgnoreObjProp(): string[]
}

查询

BaseTableDao实现了三种查询，单键、多键和分页； 查询后通过buildModelsByDBSet；将 DBSet 转换成对应的 Model

/**
 * 多键查询；传入 map，获取自动将 map 的值，转换成RdbPredicates 进行查询；
 * @param map <key:列名，value：列的值>
 */
async queryByColumns(map: Map<string, ESObject>) {
  if (this.tableConfig == undefined || this.baseDao == undefined) {
    MTRdbLog.e(TAG, "tableConfig or baseDao is undefined")
    return []
  }
  let predicates = new relationalStore.RdbPredicates(this.tableConfig.tableName);
  map.forEach((value: ESObject, key) => {
    predicates.equalTo(key, value);
  })
  return await new Promise<T[]>(resolve, _) => {
    this.baseDao?.query(predicates, (resultSet: relationalStore.ResultSet) => {
      let count: number = resultSet.rowCount;
      if (count === 0 || typeof count === 'string') {
        MTRdbLog.d(TAG, 'Query no results!');
        resolve([]);
      } else {
        const result: T[] = this.buildModelsByDBSet(resultSet);
        resolve(result);
      }
    });
  });
}
/**
 * 单键查询
 */
async queryBy(column: string, value: ValueType) {
  if (this.tableConfig == undefined || this.baseDao == undefined) {
    MTRdbLog.e(TAG, "tableConfig or baseDao is undefined")
    return []
  }
  let predicates = new relationalStore.RdbPredicates(this.tableConfig.tableName);
  predicates.equalTo(column, value);
  return await new Promise<T[]>(resolve, _) => {
    this.baseDao?.query(predicates, (resultSet: relationalStore.ResultSet) => {
      let count: number = resultSet.rowCount;
      if (count === 0 || typeof count === 'string') {
        MTRdbLog.d(TAG, 'Query no results!');
        resolve([]);
      } else {
        const result: T[] = this.buildModelsByDBSet(resultSet);
        resolve(result);
      }
    });
  });
}
/**
 * 分页查询
 * @param page 当前页
 * @param limit 获取的数量
 */
async queryByPage(page: number, limit: number) {
  if (this.tableConfig == undefined || this.baseDao == undefined) {
    MTRdbLog.e(TAG, "tableConfig or baseDao is undefined")
    return []
  }
  let predicates = new relationalStore.RdbPredicates(this.tableConfig.tableName);
  predicates.offsetAs((page - 1) * limit)
  predicates.limitAs(limit);
  return await new Promise<T[]>(resolve, _) => {
    this.baseDao?.query(predicates, (resultSet: relationalStore.ResultSet) => {
      let count: number = resultSet.rowCount;
      if (count === 0 || typeof count === 'string') {
        MTRdbLog.d(TAG, 'Query no results!');
        resolve([])
      } else {
        const result: T[] = this.buildModelsByDBSet(resultSet);
        resolve(result);
      }
    });
  });
}

buildModelsByDBSet实现：

/**
 * 将数据库查到的数据，转成 model
 * @param resultSet
 * @returns
 */
protected buildModelsByDBSet(resultSet: relationalStore.ResultSet): T[] {
  if (this.tableConfig == undefined || this.baseDao == undefined) {
  MTRdbLog.e(TAG, "tableConfig is undefined")
  return []
}
let count: number = resultSet.rowCount;
const result: T[] = [];
resultSet.goToFirstRow();
for (let i = 0; i < count; i++) {
  let columns = this.tableConfig.columnNames;
  let obj: T = this.generateModelObj() //通过接口，让具体的实现类，返回空的 Model
  this.buildSubModelByDBSet(resultSet, columns, obj, obj.getDbIgnoreObjProp())
  this.getParamsFromString(obj)
  result[i] = obj;
  resultSet.goToNextRow();
}
return result;
}

//填充具体的数据到 Model
private buildSubModelByDBSet(
  resultSet: relationalStore.ResultSet,
  columns: string[],
  parentObj: ESObject,
  ignoreObj: string[],
  embeddedPrefix?: string
) {
  let propNames: string[] = Object.getOwnPropertyNames(parentObj);
  //多层嵌套对象的属性，需要增加前缀（ "嵌套对象的命名" + "_"）对应上数据库字段
  let prefix = embeddedPrefix == undefined ? '' : `${embeddedPrefix}_`
  propNames.forEach((propName, _) => {
    let type = TSUtils.getType(parentObj, propName)
    //属性是个对象
    if (type == 'object' && !ignoreObj.includes(prefix + propName)) {
      let subObj: ESObject = TSUtils.getPropValue(parentObj, propName)
      //构建对象
      this.buildSubModelByDBSet(resultSet, columns, subObj, ignoreObj, prefix + propName)
      TSUtils.setPropValue(parentObj, propName, subObj)
    } else if (columns.includes(prefix + propName)) {
      //普通属性
      TSUtils.setPropValue(parentObj, propName,
                           this.getColumnValueByType(resultSet, parentObj, propName, embeddedPrefix))
    }
  })
}

数据库升级

在RdbCore 中获取RdbStore的时候，会进行数据库版本的判断；获取 init 时配置的数据库版本，和 rdbstore 的版本进行对比，如果需要升级，则会从BaseUpConfig中获取每个版本的升级内容，进行处理

/**
 * 检测数据库是否需要升级
 */
checkDbMigration(rdbStore: relationalStore.RdbStore) {
  try {
    MTRdbLog.d(TAG, `start checkDbMigration tableName:${this.tableName} version ${rdbStore.version}`)
    if (rdbStore.version < MTRdb.dbVersion) {
      this.addMigrations(rdbStore.version, MTRdb.dbVersion)
      this.migrationContainer.forEach(migration => {
        if (rdbStore.version <= migration.startVersion) {
          migration.migrate(rdbStore)
        }
      })
      rdbStore.version = MTRdb.dbVersion
      MTRdbLog.d(TAG, `checkDbMigration suc version ${MTRdb.dbVersion}`)
    }
  } catch (e) {
    MTRdbLog.e(TAG, `checkDbMigration fail ${e}`)
  }
}

BaseUpConfig中，定义了get allVersionInfo(): Map<number, MTRdbMigrationInfo>，由实现类返回每个版本的改变信息MTRdbMigrationInfo；然后在migration时，从所有MTRdbMigrationInfo中获取新增字段、改名字段等信息，再通过addColumnSQL,renameColumnSQL构建出相关的 sql语句，进行数据库的升级

/**
 * 基础的升级类,用于配置升级逻辑
 */
export abstract class BaseUpConfig {
  tableName: string

  constructor(tableName: string) {
    this.tableName = tableName
  }

  /**
   * 全部的版本的字段信息
   */
  abstract get allVersionInfo(): Map<number, MTRdbMigrationInfo>

  /* 略 */
  
  /**
   * 获取对应的版本
   */
  versionNewColumnCreateList(version: number, endSplit: string = "") {
    const versionMigrationInfo = this.allVersionInfo.get(version)
    const versionInfo = versionMigrationInfo?.addList
    if (!versionInfo) {
      return []
    }
    let keys = Array.from(versionInfo.entries()).map(it => {
      return `${it[0]} ${it[1]} ${endSplit}`
    })
    return keys
  }

  migrationAdd(old: number, target: number) {
    let result: string[] = []
    for (let i = old + 1; i <= target; i++) {
      const itList = this.versionNewColumnCreateList(i)
      if (itList && itList.length > 0) {
        result.push(...itList)
      }
    }
    return result
  }

  migrationRename(old: number, target: number): MTRdbRenameModel[] {
    let result: MTRdbRenameModel[] = []
    for (let i = old + 1; i <= target; i++) {
      const itList = this.versionRenameColumns(i)
      if (itList && itList.length > 0) {
        result.push(...itList)
      }
    }
    return result
  }

  /**
   * 获取对应的版本
   */
  versionRenameColumns(version: number) {
    const versionMigrationInfo = this.allVersionInfo.get(version)
    const versionInfo = versionMigrationInfo?.modifyList
    if (!versionInfo) {
      return []
    }
    return versionInfo
  }

  /**
   * 升级逻辑
   * @param oldVersion
   * @param targetV
   * @returns
   */
  migration(rdbStore: relationalStore.RdbStore, oldVersion: number, targetV: number) {
    //执行sql
    const tabName = this.tableName
    //新增字段
    const addList = this.migrationAdd(oldVersion, targetV)
    MTRdbLog.i(TAG, 'migration addList:' + JSON.stringify(addList));
    if (addList && addList.length > 0) {
      this.exeSQLSync(rdbStore, addList, (it: string) => {
        const sql = this.addColumnSQL(tabName, it)
        MTRdbLog.i(TAG, '新增字段.' + sql);
        return sql
      })
    }
    // 修改字段
    const renameList = this.migrationRename(oldVersion, targetV)
    MTRdbLog.i(TAG, 'migration renameList:' + JSON.stringify(renameList));
    if (renameList && renameList.length > 0) {
      this.exeSQLSync(rdbStore, renameList, (it: MTRdbRenameModel) => {
        const sql = this.renameColumnSQL(tabName, it.toSQLString())
        MTRdbLog.i(TAG, '修改字段.' + sql);
        return sql
      })
    }
  }

  /**
   * 同步执行批量的SQL
   * @param rdbStore 库
   * @param list 批量处理的字段数据
   * @param sqlBuild 构建sql的语句
   */
  exeSQLSync<T>(rdbStore: relationalStore.RdbStore, list: T[], sqlBuild: (it: T) => string) {
    // 修改字段
    for (let i = 0; i < list.length; i++) {
      const it = list[i]
      const sql = sqlBuild(it)
      rdbStore.executeSync(sql)
    }
  }

  addColumnSQL(table: string, column: string, columnType?: string) {
    const cType = columnType ?? ""
    return `ALTER TABLE ${table} ADD COLUMN ${column} ${cType}`
  }

  renameColumnSQL(table: string, columnSql: string) {
    return `ALTER TABLE ${table} RENAME COLUMN ${columnSql} `
  }
}

MTRdb 基本使用方式

基本使用

• 下载 ohpm install mtrdb
• 初始化

//配置数据库
MTRdb.init({
  application: this.context.getApplicationContext(), //applicationcontext
  rdbConfig: {
    name: 'DEMO_DB', //数据库名
    securityLevel: relationalStore.SecurityLevel.S1
  },
  dbVersion: 1, //数据库版本
  enableLog: true, //是否开启log
  logLevel: hilog.LogLevel.INFO //log 输出等级
})

• 实现表接口类BaseTableInterface

export class XXTableConfig implements BaseTableInterface {
  /**
* 表名
*/
  tableName = `TABLE_NAME`
  /**
 * 表创建语句
 */
  sqlCreate: string = `CREATE TABLE IF NOT EXISTS ……`
  /**
 * 最新的表列名（多层嵌套对象的属性，需要增加前缀（ "嵌套对象的命名" + "_"）对应上数据库字段）
 */
  columnNames: string[] = [`id`, `tab`]
  /**
* 表主键，支持多主键
*/
  uniqueColumnName: string[] = ['id', 'tab']

• 继承BaseTableDao BaseTableDao<T extends BaseModel>

export class XXBeanDao extends BaseTableDao<XXBean> {
  saveParamsToString(model: XXBean): void {
    //高级用法，复杂嵌套对象，只需要数据时，可以直接 string  化；
    //以 string 形式，保存的的对象，一般直接JSON.stringify
  }

  getParamsFromString(model: XXBean): void {
    //从 string 还原 数据，填充到 model
  }

  generateModelObj(): XXBean {
    return new XXBean()
  }
}

• 实现BaseModel BaseModel,支持Sendable

@Sendable 
export class XXBean implements BaseModel {
  id: number = 0;
  tab: string = '';

  constructor(id:number,tab:string){
    this.id = id
    this.tab = tab
  }

  getUniqueProp(): Map<string, ESObject> {
    return new Map<string, ESObject>([['id', this.id], ['tab', this.tab]])
  }
  getDbIgnoreObjProp(): string[] {
    return ['ignore_prop']
  }

• 初始化Dao对象，建议做缓存

export class DemoDB {
private static dao: XXBeanDao

public static async getXXDao(): Promise<XXBeanDao> {
return new Promise(resolve => {
  if (DemoDB.dao) {
    resolve(DemoDB.dao)
    return
  }
  DemoDB.dao = new XXBeanDao(new XXTableConfig(), () => {
    resolve(DemoDB.dao)
  })
})
}
}

• new Dao()时，就会自动初始化对应的数据表；之后通过 Dao 进行数据对象的操作即可。

DemoDB.getXXDao().then(dao => {
dao.insert(new XXBean()).then(result=>{
  //todo
})
})

数据库升级操作

初始化数据库版本改为 2

TableInterface实现类增加升级配置updateConfig

配置升级内容：

export class MaterialUpConfig extends BaseUpConfig {
  get allVersionInfo(): Map<number, MTRdbMigrationInfo> {
    return this.allMap;
  }
  //新增 topic 列
  addList: MTRdbColumn = new Map([
    ["topic", ColumnType.TEXT]
  ])
  //列名is_with_filter 改为 extra_info_is_with_filter
  renameList: MTRdbRenameModel[] = [
    new MTRdbRenameModel("is_with_filter", "extra_info_is_with_filter")
  ]
  /*版本2 对应的修改配置 */
  version2: MTRdbMigrationInfo = {
    table: this.tableName,
    addList: this.addList,
    modifyList: this.renameList
  }
  /**
   * Map<版本，版本对应的差异 MigrationInfo>
   */
  allMap: Map<number, MTRdbMigrationInfo> = new Map([
    [2, this.version2]
  ])
}

TableInterface实现类修改sqlCreate和 columnNames

sqlCreate: string = `CREATE TABLE IF NOT EXISTS ${this.tableName} (
    material_id                     INTEGER,
    tab_id                          TEXT    DEFAULT (''),
    extra_info_is_with_filter       INTEGER,   
    ${this.updateConfig.allNewColumnCreate(",")}
    PRIMARY KEY (material_id,tab_id)
    )` /*（is_with_filter 改名成 extra_info_is_with_filter）*/


/**
 * 最新的表列名  （is_with_filter 改名成 extra_info_is_with_filter）
 */
// columnNames: string[] =this.columnNameOrigin
columnNames: string[] =
  [`material_id`, `tab_id`, `extra_info_is_with_filter`, ...this.updateConfig.allNewColumnList()]

在下次初始化 Rdb 时，进行数据库升级检测，会自动升级数据库。完成后，如下图