Hypersistence blog - Key takeaways 2


CDC(Change Data Capture)

*CDC(change data capture) works effectively by decoding the underlying transaction log and extract the CDC events from it. The extracted events are propagated to Apache Kafka, which further exchange messages b/w various other modules of a large enterprise system.


 *Envers is the hibernate native implementation for CDC. change events triggered outside of the persistence captured will not be captured by Envers. AuditReaderFactory class takes a JPA EntityManager or Hibernate Session to generate an AuditReader object.

Connection Pooling

*Opening/closing connections are quite an expensive operation. The most obvious reasons for reusing a database connection would be:
  • reducing the application and database management system OS I/O overhead for creating/destroying a TCP connection
  • reducing JVM object garbage

*To avoid system overloading we use throttling. Whenever the number of incoming requests surpasses the available resources, we basically have two options:
  • discarding all overflowing traffic, therefore decreasing availability
  • queuing requests and wait (for as long as a timeout threshold) for busy resources to become available

*If there is a traffic spike the connection pool will level it instead of saturating all available database resources.

 *The waiting step and the timeout mechanism are safety hooks, preventing excessive database server load. If one application gets way too much database traffic, the connection pool is going to mitigate it, therefore, preventing it from taking down the database server (hence affecting the whole enterprise system).

 *To configure a pool we may need to pay attention to many pool settings such as:
  • minimum size
  • maximum size
  • max idle time
  • acquire timeout
  • timeout retry attempts
*Sizing connection pools are not an upfront design decision. In large enterprise systems, you need adaptability and monitoring is the first step to taking the right decisions.

N+1 problem

*The N+1 query issue happens when you forget to fetch an association and then you need to access 

 List<PostComment> comments = entityManager.createQuery(
    "select pc " +
    "from PostComment pc " +
    "where pc.review = :review", PostComment.class)
.setParameter("review", review)
.getResultList();

 for(PostComment comment : comments) {
    LOGGER.info("The post title is '{}'", comment.getPost().getTitle());
}
Because the post association is not initialized, Hibernate must fetch the Post entity with a secondary query, and for N PostComment entities, N more queries are going to be executed (hence the N+1 query problem).

Comments

Post a Comment

Popular posts from this blog

Distributed database design using CAP theorem

Image
CAP is a theorem/standard which can be applied while designing distributed systems like distributed databases. A distributed database is a set of databases(nodes) which are spread across the network.  These set of databases(nodes) in the distributed database can be either designed as either a  master/slave  or a  sharded model . Consistency -  Database should return the latest write on the DB. Availability -  Database should return data in a decent amount of time .  But, the data might not be latest. Partition tolerant - I n spite of n/w failure, the database should always be available. A distributed database can be designed either as a CP, AP, CA but not with all the 3 capabilities.  AP - Available and Partition tolerant  -  Sharded model (Data is distributed, No Master)  -      Availability  - Because, the data is distributed, any of the nodes can return the data. So, always availabl...
Continue Reading »

SQL Analytical Functions - Partition by (to split resultset into groups)

Partition by:  Splits the resultset into groups.  Employees -Table EMPLOYEE_ID FIRST_NAME SALARY ----------- -------------------- ---------- 100 Steven 24000 101 Neena 17000 102 Lex 17000 103 Alexander 9000 104 Bruce 6000 105 David 4800 106 Valli 4800 107 Diana 4200 select e.employee_id, e.department_id, salary, row_number() over (partition by department_id order by salary desc) as rownumber from employees e; The above query splits the result into small subgroups for every department.  Syntax :  Function(arg1,..., argn) OVER ( [PARTITION BY <..>] [ORDER BY <....>]  [<window_clause>] ) partition by  - create  subgroups for e...
Continue Reading »

Easy approach to work with files in Java - Java NIO(New input output)

  NIO - New IO package. Introduced in 1.4. It is buffer based and non-blocking. Hence, faster than IO package. It supports symbolic links, posix file attributes(works only on OS's which are POSIX compliant), ACL (Access control list). Paths and Files are the main classes Creating a potential file reference: Path path = Paths.get("c:/demo/"); //possible file reference Creating a directory, file, write to the file, read from the file: //create all the non existing directories in the path. Ignores creating a directory, if that directory already exists Path directories = Files.createDirectories("c:/demo/example1/properties/"); Path newFilePath = Files.createFile("c:/demo/README.txt"); //Creating a file //Writting bytes to the file Files.write(newFilePath, "This is a dummy text".getBytes(StandardCharsets.UTF_8)); //Reading bytes from file String readFile = new String(Files.readAllBytes(newFilePath), "UTF-8"); Files.readAllBytes reads all...
Continue Reading »