BatchJS-Data

Extension of Batch JS adding data storage support for databases.

---

• BatchJS-Data
• Download
  • NPM
  • Yarn
• Usage
• Documentation
• Collaborators welcome!

---

Download

NPM

npm install batchjs-data
npm install mariadb         #For MariaDB implementation
npm install mysql2          #For MySQL implementation
npm install pg @types/pg    #For PostgreSQL implementation

Yarn

yarn add batchjs-data --no-optional
yarn add mariadb         #For MariaDB implementation
yarn add mysql2          #For MySQL implementation
yarn add pg @types/pg    #For PostgreSQL implementation

Usage

1. Create your reader

   import { DatabaseSync } from "node:sqlite";
   import { SqliteBatchEntityReader } from "batchjs-data/sqlite";
   import { UserDTO } from "./UserDTO";
   
   export class UserBatchReader extends SqliteBatchEntityReader<
     UserDTO,
     UserDTO
   > {
     constructor(options: { batchSize: number; query?: string }) {
       super({
         batchSize: options.batchSize,
         dbConnectionFactory: async () =>
           Promise.resolve(new DatabaseSync("./database.db")),
         query: options.query || "SELECT id, username FROM users",
         rowToEntity: (row: UserDTO) => row,
       });
     }
   }

2. Create your writer

   import { DatabaseSync, StatementSync } from "node:sqlite";
   import { SqliteBatchEntityWriter } from "batchjs-data/sqlite";
   import { UserDTO } from "./UserDTO";
   
   export class UserBatchWriter extends SqliteBatchEntityWriter<UserDTO> {
     constructor(options: { batchSize: number }) {
       super({
         batchSize: options.batchSize,
         dbConnectionFactory: async () =>
           Promise.resolve(new DatabaseSync("./database.db")),
         prepareStatement: "INSERT INTO users (id, username) VALUES (?, ?)",
         saveEntity: async (entity: UserDTO, stmt: StatementSync) => {
           stmt.run(entity.id, entity.username);
         },
       });
     }
   }

3. Use them in your BatchJS Job

   import { Job, Step } from "batchjs";
   
   // Implement a step
   class StepImplementation extends Step {
     // Set a name to the step
     constructor(name: string = "DemoStep") {
       super(name);
     }
   
     // Implement the reader to load step data source
     protected _reader() {
       return new UserBatchReader({ batchSize: 2 });
     }
   
     // Implement the processors to transform data sequently using our streams or your own streams
     protected _processors() {
       const opts: TransformOptions = {
         objectMode: true,
         transform(
           chunk: unknown,
           encoding: BufferEncoding,
           callback: TransformCallback,
         ) {
           this.push(chunk);
           callback();
         },
       };
       return [new Transform(opts), new Transform(opts)];
     }
   
     // Implement the write to stock final step data
     protected _writer() {
       return new UserBatchWriter({ batchSize: 2 });
     }
   }
   
   // Implement a Job
   class JobImplementation extends Job {
     // Implement to set the steps to be sequently executed.
     protected _steps() {
       return [new StepImplementation(), new StepImplementation()];
     }
   }
   
   // Instance the Job
   const job = new JobImplementation("My job");
   
   // Set events listener
   job.on("stepStart", (step: step) => {
     console.log(`Starting step ${step.name}`);
   });
   
   // Launch the job
   job
     .run()
     .then(() => {
       console.log("Job completed successfully");
     })
     .catch((error) => {
       console.log("Job completed with errors");
     });

Documentation

• Documentation site
• Core API
• MariaDB API
• MySQL API
• PostgreSQL API
• SQLite API

To preview the documentation locally, run:

npm run docs

Collaborators welcome!

• ¿Do you like the project? Give us a ⭐ in GitHub.
• 🆘 ¿Do you need some help? Open a discussion in GitHub help wanted
• 🐛 ¿Do you find a bug? Open a issue in GitHub bug report
• 💡 ¿Do you have a great idea? Open a issue in GitHub feature request
• 💻 ¿Do you know how to fix a bug? Open a pull request in GitHub pull request.
• ¿Do you know a security issue? Take a read to our security strategy.

Subscribe our code of conduct and follow the Contribution Guidelines.