Instructor: 3 YOE, working in Adobe Sahil Yadav Topic: HLD Conceptual class
Notes
- Today we will design a web crawler, starting with Cap theorem. Cap theorem works for only distributed databases.
- Mentionedmaster_slave architecture for above.
- Consistency Whenever we read we should the data of latest write.
- Availability Always data should be available, doesn’t matter old data or new data but it should be available.
- Partition_Tolerance Nodes (Hard disks) should be connected, in below example if the leftmost nodes gets disconnected then all network will get disconnected,
- taskquestion Which algorithm is used to determine connected Graph? ^ncpd
- taskquestion single node data, why is availability considered here, since it is not a distributed system.
- Consistency_n_Partition_Tolerant - Example: MySQL, PostgreSQL etc.
- Partition_Tolerance_n_Availability - Example: Dynamo DB
For a device accessing local DB, android application having its own local DB, how do CAP theorem apply (or the case of one node). It can be considered asConsistency_n_Partition_Tolerant Still how? In case the phone gets broken.
back_of_the_envelope estimation:
- Client → Web browser → DNS server → Gets IP → Hit Load Balancer of the server → Server node, DB etc.
- L1_Cache → 05nanosec
- L2_Cache →
- Q. How much time it will take to transfer 1KB of data over 1GBPS connection?
- Ans. 1GB data in 1 sec, 1/1000 sec (1 nanosec)
- work_philosophy No need to try to much to get manager trust, focus on your technical knowledge and Personal Branding.
- Talked on Sivi today, this should be good for practice as well.talk_with_strangers, learn to manage and do balanced talks even if the other person is rude or has attitude issue. Love them, just question them more.
Links
- https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-cp-or-ap.html
- Brew Eric wrote blog on Google Spanner in practical way.
- Good link forback_of_the_envelope → https://systemdesign.one/back-of-the-envelope/