The docker compose file I use:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

version: '3'

services:
  mysql:
    image: mysql/mysql-server
    container_name: mysql
    environment:
      MYSQL_ROOT_PASSWORD: <mypw>
      MYSQL_DATABASE: app_db
      MYSQL_USER: dev
      MYSQL_PASSWORD: <mypw>
    ports:
      - "3306:3306"
    volumes:
      - ./data:/var/lib/mysql
  phpmyadmin:
    image: phpmyadmin/phpmyadmin
    container_name: pma
    links:
      - mysql
    environment:
      PMA_HOST: mysql
      PMA_PORT: 3306
      PMA_ARBITRARY: 1
    restart: always
    ports:
      - 8081:80

Access into the docker container

docker exec -it <container id> bash

The container id can be find by docker ps

Create new database

mysql -u root -p

CREATE DATABASE 'newdatabase';

1
2
3
4

CREATE USER 'dev'@'localhost' IDENTIFIED BY 'mypw';
CREATE USER 'dev'@'%' IDENTIFIED BY 'mypw';
GRANT ALL ON *.* TO 'dev'@'localhost';
GRANT ALL ON *.* TO 'dev'@'%';

flush privileges;

CREATE USER 'newuser'@'%' IDENTIFIED BY 'newpassword';

Then give the new account “newuser” permission to read and write the new database

GRANT ALL PRIVILEGES ON newdatabase.* TO 'newuser'@'localhost';

quit the root login and switch to the user

quit

mysql -u <newuser> -p

First, create a `hello-world.js’ file

1

console.log("Hello World")

Then, create a Dockerfile in the same directory, the Dockerfile should look like that:

1
2
3
4
5
6
7
8
9
10

FROM ubuntu

RUN apt update && apt install nodejs -y

WORKDIR /app

COPY . .

CMD ["node", "/app/hello-would.js"]

FROM ubuntu means we ubuntu the ubuntu image as the base

RUN apt update && apt install nodejs -y means install nodejs into the ubuntu image

WORKDIR is used to define the working directory of a Docker container.

COPY . . is to copy the current directory(the hello-world.js) to the WORKDIR

CMD ["node", "/app/hello-would.js"] is the command to run after the image load.

Finally, Build the image

docker build -t {image name and version} .

-t is the tag for the image name and version, for example, etklam/hello_app:0.1

. is the Dockerfile directory

Main different

The main different between utf8 and utf8-mb4 is utf8 is only 3 bytes but utf8-mb4 is 4bytes instead. Note that UTF8 have 4 bytes, utf8mb4 is the true utf8 character set.

In general, it is enough to use MySQL utf8 to set up a website, however, Some special (Chinese) characters or common emoji are not included in 3 bytes. Therefore, most Chinese characters are sufficient, but they cannot be used for all characters. If you want to use special characters or emoticons, you cannot use MySQL’s utf8 character set to store them.

UTF8 to UTF8mb4 does not cause incompatibility problems.

Create XMLHttpRequest object

XMLHttpRequest is basic of Ajax request,

1

const xhr = new XMLHttpRequest();

Send a request to the server

1
2

xmlhttp.open("GET", "url"); // url is the api endpoint
xml.send();

onreadystatechange

There are 5 state of xmlhttprequest, from 0 to 4

0. Request not initialized
1. Server connection is established
2. Request has been received
3. Request is being processed
4. Request completed and response is ready

Hence, When readyState is equal to 4 and the status is 200, the response is ready.

1
2
3
4
5

xml.onreadystatechange=function() {
    if(xml.readyState==4 && xml.status==200) {
        document.getElementById("myDiv").innerHTML=xml.responseText;
    }
}

Ajax json examples

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

function loadXMLDoc(){
    // init a xmlhttprequest
    const xmlhttp = new XMLHttpRequest();

    // init a onreadystatechange and parse xml to json 
    xmlhttp.onreadystatechange=function()
    {
        if (xmlhttp.readyState==4 && xmlhttp.status==200)
        {
        var myArr = JSON.parse(this.responseText);
        myFunction(myArr)
        }
    }
    //send the xml request
    xmlhttp.open("GET","/try/ajax/json_ajax.json",true);
    xmlhttp.setRequestHeader("Content-Type", "application/json;charset=UTF-8");
    xmlhttp.send();
}
// function to change the dom
function myFunction(arr) {
  var out = "";
  var i;
  for(i = 0; i < arr.length; i++) {
    out += '<a href="' + arr[i].url + '">' + 
    arr[i].title + '</a><br>';
  }
 document.getElementById("myDiv").innerHTML=out;
}

Ajax with Promise examples

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

const p = new Promise((resolve, reject)=>{
    const xmlhttp = new XMLHttpRequest();
    xmlhttp.onreadystatechange=function()
    {
        if (xmlhttp.readyState==4){
            if (xmlhttp.status==200)
            {
                //success
                resolve(this.responseText)
            }else{
                //fail
                reject(xmlhttp.status)
            }
        }
    }
})

//promise followup
const result = p.then(function(value){
    let myArr = JSON.parse(value)
    myFunction(myArr)
    return myArr
},function(reason){
    console.log(reason)
    return 'error'
})

// function to change the dom
function myFunction(arr) {
  var out = "";
  var i;
  for(i = 0; i < arr.length; i++) {
    out += '<a href="' + arr[i].url + '">' + 
    arr[i].title + '</a><br>';
  }
 document.getElementById("myDiv").innerHTML=out;
}

Spring initialization

• spring-boot-starter-web
• spring-boot-starter-data-jpa
• spring-boot-devtools
• mysql-connector-java
• lombok

Connect to database

in the resource/application.properties, we can config the hibernate connection for mysql

1
2
3
4
5
6

spring.jpa.hibernate.ddl-auto=create-drop
spring.datasource.url=jdbc:mysql://localhost:3306/mydb
spring.datasource.username=username
spring.datasource.password=password
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.jpa.show-sql= true

Simple API

To build a simple api, we need to add two annotaion to the main

• RestController
• @GetMapping/@PostMapping/ others

1
2
3
4
5
6
7
8
9
10
11
12
13

@SpringBootApplication
@RestController
public class PpfaSpringApplication {

 public static void main(String[] args) {
  SpringApplication.run(PpfaSpringApplication.class, args);
 }

 @GetMapping
 public String hello() {
  return "Hello World";
 }
}

First, install jenv by using Homebrew

1

brew install jenv

After that, need to config the zsh

1
2
3

echo 'export PATH="$HOME/.jenv/bin:$PATH"' >> ~/.zshrc
echo 'eval "$(jenv init -)"' >> ~/.zshrc
source ~/.zshrc

It only found the system default Java:

1

jenv versions

Add the jdk you installed to the jenv. Personally my jdks are installed at /Users/klam/Library/Java/JavaVirtualMachines/

For example:

jenv add /Users/klam/Library/Java/JavaVirtualMachines/azul-17.0.3/Contents/Home/

jenv global 17 to swap between different jdk for the default jdk

you can also use jenv local 17 to specifies the Java version of a folder

In Java, there are two main types of problems that can occur during the execution of a program: Errors and Exceptions.

1. Errors

Errors are serious issues that occur beyond the control of the application. These are typically problems related to the Java Virtual Machine (JVM) itself, such as:

• StackOverflowError
• OutOfMemoryError
• VirtualMachineError

These errors are often unrecoverable and should not be handled in the code. When an error occurs, it’s best to let the system crash or shut down gracefully, as the environment may no longer be stable.

2. Exceptions

Exceptions are issues that arise during the normal operation of a program and can usually be anticipated and handled. For example:

• Trying to read a file that doesn’t exist.
• Invalid user input.
• Attempting to divide by zero.

Java provides a robust mechanism to handle exceptions using try-catch-finally blocks. Exceptions are further categorized into two types:

a. Checked Exceptions

These are exceptions that are checked at compile time. The compiler requires the developer to handle these exceptions explicitly, either by using a try-catch block or by declaring them in the method signature using the throws keyword.

Examples:

• IOException
• SQLException

b. Unchecked Exceptions

These are exceptions that are not checked at compile time. They usually indicate programming bugs, such as logic errors or improper use of an API. These exceptions inherit from RuntimeException.

Examples:

• NullPointerException
• ArrayIndexOutOfBoundsException
• IllegalArgumentException

Summary

Understanding the difference between errors and exceptions—and between checked and unchecked exceptions—helps in writing more robust and fault-tolerant Java applications.

Let me know if you’d like a more casual tone or if you want to turn this into a tutorial-style post!

Introduction

I am working with a login api, and therefore I have some notes about Session and Token (JWT - Json web token).

Session

The general practice of a login system should be to verify that the customer’s login information is correct. Then add a logged in attribute to the client’s session if it is correct. There are usually some tools that help us doing that. Generally the default name of the session(cookie) is “JSESSIONID”; Stored in the client’s cookie, so we don’t have to write any more complicated operations in the program.

Each time the Client Side send a request, we bring the session id along with it. Server side will take the session ID and find out the specific session from the many sessions stored in Server.
There it is, if there are 10000 user online, server need to store 10000 different session in the database. Which is a very high IO, also, there is also the problem of how to share sessions between multiple hosts.

To solve this problem, we normally use Redis.

JWT token

It is very popular to use JWT as a Token instead of session. jwt is a string encrypted by the server and issued to the client.
After receiving the token, the client sends a request with the token in case of need, so that the Server can decrypt and verify the identity.
Because the token itself stores the authentication information of the client. In general, the Server will no longer store the token after it is issued.
Note that, the token can actually be stored in a cookie.

JWT implementation

There are three part of a JWT, header, payload, signature

The whole thing will use base64 encode

Header

• alg: Cryptographic algorithms used
• typ: JWT

Payload

• iss: Issuer
• sub: subject, can be the key value such as account no.
• exp: expiration time

Signature

sign(hash(header+payload))

The signature also certifies that only the party holding the private key is the one that signed it.

Generating JWT

1
2
3
4
5
6
7
8
9

// JWT code here
Date expireDate = new Date(System.currentTimeMillis()+30*60*1000);
                    
String jwtToken = Jwts.builder().setSubject(email)
    .setExpiration(expireDate)
    .signWith(SignatureAlgorithm.HS512, "secret")
    .compact();
                    
return jwtToken;

Check the token

• notes 1 : Whenever the user wants to access a protected route or resource, the user agent should send the JWT, typically in the Authorization header using the Bearer schema. The content of the header should look like the following: Authorization: Bearer <token>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

public class AuthorizationCheckFilter extends OncePerRequestFilter{

    @Override
    `protected void doFilterInternal(HttpServletRequest req, HttpServletResponse res, FilterChain chain) throws ServletException, IOException {
                if(!req.getServletPath().equals("/api/v1/user/login")){
                    
                    
                    String authorHeader =  req.getHeader(AUTHORIZATION);
                    String bearer ="Bearer "; // notes 1

                    if(authorHeader!= null && authorHeader.startsWith(bearer)){
                        try{
                        String token = authorHeader.substring(bearer.length());
                        Claims claims = Jwts.parser().setSigningKey("MySecret")
                        .parseClaimsJws(token).getBody();

                        System.out.println("JWT payload:"+claims.toString());

                        chain.doFilter(req, res);
                        
                        }catch(Exception e){
                            System.err.println("Error : "+e);
                            res.setStatus(FORBIDDEN.value());
                            
                            Map<String, String> err = new HashMap<>();
                            err.put("jwt_err", e.getMessage());
                            res.setContentType(APPLICATION_JSON_VALUE);
                            new ObjectMapper().writeValue(res.getOutputStream(), err);
                        }
                    }else{
                        res.setStatus(UNAUTHORIZED.value());
                    }
                }else{
                    chain.doFilter(req, res);
                }
        
    }

}

The jwt implementation of nestjs
https://github.com/etklam/nestjs-jwt-implementation

This notes is the learning process of Spring boot. Follow the https://www.udemy.com/course/spring-hibernate-tutorial/learn/lecture/12940996#overview

Connect to database

in the Resource path, there are a application.properties

input the JDBC properties

1
2
3
4
5
6

#
# JDBC properties

spring.datasource.url=jdbc:mysql://localhost:3306/employee_directory?useSSL=false&serverTimezone=UTC
spring.datasource.username=<username>
spring.datasource.password=<assword>

Create Entity

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

@Entity
@Table(name="employee")
public class Employee {

    // define fields

    @Id
    @GeneratedValue(strategy=GenerationType.IDENTITY)
    @Column(name="id")
    private int id;

    @Column(name="first_name")
    private String firstName;

    @Column(name="last_name")
    private String lastName;

    @Column(name="email")
    private String email;


    // define constructors

    public Employee() {

    }

    public Employee(String firstName, String lastName, String email) {
        this.firstName = firstName;
        this.lastName = lastName;
        this.email = email;
    }

    //define getter setter toString...
}

Hibernate implementation

Create a interface in src/java/projectname/dao/EmployeeDAO

DAO aka Data Access Object, is the “model” of MVC.

1
2
3
4
5
6
7
8
9
10
11

public interface EmployeeDAO {

    public List<Employee> findAll();

    public Employee findById(int theId);

    public void save(Employee theEmployee);

    public void delete(Employee theEmployee);

}

Implementing the interface in src/java/projectname/dao/EmployeeDAOHibernateImpl

@Repository

contains the api to control the database;
-. createQuery(…)

• .get(…)
• etc…

@Transactional

transaction is atom unit of the DBMS, Provides a way for database operation sequences to recover from failure to a normal state.
DBMS needs to ensure that all operations in the transaction are completed successfully and the results are permanently stored in the database.
IF some operations in the transaction are not completed successfully, all operations in the transaction need to be rolled back to the state before the transaction has no effect on the database or the execution of other transactions, and all transactions needs to be executed independently.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

// repository contains the api to control the database;
@Repository
public class EmployeeDAOHibernateImpl implements EmployeeDAO {

    // define field for entitymanager
    private EntityManager entityManager;

    // set up constructor injection
    @Autowired
    public EmployeeDAOHibernateImpl(EntityManager theEntityManager) {
        entityManager = theEntityManager;
    }


    @Override
    @Transactional
    public List<Employee> findAll() {

        // get the current hibernate session
        Session currentSession = entityManager.unwrap(Session.class);

        // create a query
        Query<Employee> theQuery =
                currentSession.createQuery("select e from Employee e", Employee.class);

        // execute query and get result list
        List<Employee> employees = theQuery.getResultList();

        // return the results
        return employees;
    }

    @Override
    public Employee findById(int theId) {
        Session currentSession = entityManager.unwrap(Session.class);

        Employee theEmployee = currentSession.get(Employee.class, theId);
        return theEmployee;
    }

    @Override
    public void save(Employee theEmployee) {
        Session currentSession = entityManager.unwrap(Session.class);

        currentSession.saveOrUpdate(theEmployee);
    }

    @Override
    public void delete(Employee theEmployee) {
        Session currentSession = entityManager.unwrap(Session.class);

        currentSession.delete(theEmployee);
    }

}

Service

Mostly similar to the dao, first create a interface, then implementing the interface.

1
2
3
4
5
6
7
8
9
10

public interface EmployeeService {

    public List<Employee> findAll();

    public Employee findById(int theId);

    public void save(Employee theEmployee);

    public void deleteById( int theId);
}

1
2
3
4
5

@Service
public class EmployeeServiceImpl implements EmployeeService {
...
}

Rest

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

@RestController
@RequestMapping("/api")
public class EmployeeRestController {
    private EmployeeService employeeService;

    @Autowired
    public EmployeeRestController(EmployeeService theEmployeeService) {
        employeeService = theEmployeeService;
    }

    @GetMapping("/employees")...
    @PostMapping("/employees")...
    @GetMapping("/employees/{employeeId}")
    @PutMapping("/employees")
    @DeleteMapping("/employees/{employeeId}")
    ...
}

Cryptography

• Symmetric encryption (base on substitution and permutation): a secret key and an encryption algorithm
• Asymmetric encryption (base on exponential): public key and private key system.
• Public-key: provide confidentiality(encryption/decryption) and authentication (signature)
• Both of them are secure and useful in diff scenarios
• RSA algorithm is based on the difficulty of factoring problem.
• Hash function can be applied to any sized message and produce fixed length message digest.
• Sign the message digest instead of the message itself.
  • first hash the message
  • Then encrypt(sign) the hashed value
  • The message usually longer than the key size
• DSA signature is based on the difficulty of discrete logarithms problem.

AES

permutation (shift rows)

AES key will be expanded(x11)
16bytes -> 176bytes
The chiper consists of N rounds, N depends on the key length:

• 16bytes: 10rounds
• 24bytes: 12rounds
• 14bytes: 14rounds

Certification Authority (CA)

Sign certificate that bind subscriber’s name and his public key.
Indicates that the subscriber has sole control and access to the corresponding private key.

Public Key Infarstructure (PKI)

A set of policy, processes, server platforms, software etc…
to administer certificates

• issue
• maintain
• revoke

Bitcoin

• Merkle Tree Root is public for Verification
• Merkle Tree Root for txs, and store in the block header
• non-singular elliptic curve is the set of points and the point at infinity O
• The point at infinity O is the identity elements
• Bitcoin use Mudulo p - secp256k1, ECDSA

Bitcoin structure

Header:

• Version Number
• Hash of prev block header (by SHA256 double hash)
• Hash of Transactions(merkle tree)
• timestamp
• Threshold(difficulty)
• Nonce any value

Body:

• Number of TXs
• Coinbase
• Regulars etc

Hash function Requirment

• Easy to compute but diffcult to invert
• Collision resistant

New diffculty caluate

Bitcoin create once every 10 minutes
Update the diffculty every 2016 blocks
T-new = T-sum/(2016*10*60) * T

Simple Payment Verification Node (SPV)

only stores the block header, contact full nodes when information needed.

Mining

read my code

Probability that the block hash falls below the target threshold T:
p= T+1/2^256

Lock Time

>= 510^8, it is a Unix time
< 510^8, it is a block blockHeight

• Bitcoin prevents double spending(verifty every single node) and tampering(unless 51%)
• Mining difficulty adjucted to regulate coin supply
• Bitcoin address are shared over the internet

ETH

Account based model instead of UTXO. Main a global state to record the account balance

Accounts in ETH

Externally owned Accounts(EOA)

• Controlled by private key
• Has an Ether balance
• no code

Contract Accounts

• Has balance
• Has code (smart contract)
• has own permanent state

ETH Contact Transaction

1. Create new contract
2. Message the contract to execute it

Gas fee, Gas limit

Gas fee is the price per gas unit. Different operation cost different unit of gas.
Gas limit is the most you are willing to paid. The remainer will refund

ETH storage management - Radix Trie and Patricia Trie

—— Skip ——

• State Trie
• Transaction Trie
• Receipts Trie

ETH consensus - simpler GHOST

Uncle Block: floked block. Give reward to honest but unlucky minor.

intrinsic reward = 5

If include a uncle block, minor can get extra 1/32 intrinsic reward.
Uncle can get depands on block height
(Uncle + 8 - block that include uncle)* intrinsic reward/8

Incentive: reward unluck but honest miners. Make it more fair.

Solidity simplest form notes

• require The require function call defines conditions that reverts all changes if not met
• emit an event after successful money transfer

Token

can be programmed to provide different functions

Initial Coin Offering (ICO)

raise funds for a company to create a new coins. similar to IPO

Consensus

PoW

• the longest chain wins
• the one growing fastest will be the longest and most trustworthy
• take a lot of time to generate a block
• if too easy, the chain can be DDoS attack
• Huge Energy Consumption

PoS

• creator of a block chosen in a random way, depending on the user’s wealth
• In order to validate, forger must first put their own coin at “stake”.
• When folk, pos vote

Randomized Block Selection

randomizaion to generate the following forger. Not true random in computer world. So is usually able to predict which user will be selected to forge the next block.

Coin Age Based selection

coin age = time * amount

Target * CoinAge = the hash difficulty.

To join the PoS, you might load your coins to other or join the pool youself.

Nothing at stake

when folk, vote for both because it gains most benefits. Always win, nothing to lose.
The blockchain might never reach Consensus
Use casper: punish

DPoS Delegated Proof of stake

vote to elect witnesses
21-100 elected witnesses in a DPoS. time slots are given to each witness to publish their block.
Longest chain wins
Much faster than POW and POS

Byzantine Fault Torlerance

1. Commander -> all traitor
2. traitor boardcast -> other traitor
3. consensus

Consensus if at least 3m+1 nodes can achieve consensus. M is malicious nodes

Oral Message

1. All messages are delivered correctly
2. Know who this message is from
3. Missing messages can be detected

Permissioned Blockchain

• Regulation
• Complete control of their data
• Can be fully centralizated. Members negotiate.

Membership Service Providers (MSPs)

PKI and CA

Endorsement and validation policy can be adjusted as need.

1. Client -> Endorser (proposal)
2. Endorser check the certificate and others to validate the transactions
3. Executes the chaincode
4. Enderser -> Client
5. Client -> Orderer
6. Orderer include the transaction and generate blocks
7. Orderer -> Anchor
8. Anchor -> boardcast the block
9. Peer verify the new block
10. Peer -> Client

Channel

Maintan a Ledger, only nodes in channel can access this ledger.

Collection

The data on the chain can be said to be “permanent” and “public” to be shared among the participants. However, in the real application scenario, many data are not “publicly” stored in the blockchain due to privacy.
Only the header is stored on chain for verification.