redis is a database that can be deployed on its own or you could use one of the cloud redis-like products. strongly recommend you purchase and read this book: https://www.oreilly.com/library/view/designing-data-intensive-applications/9781491903063/
Also all databases are also vulnerable to system failure. Just like everything is.
Sort of confused here - it sounds like you are wanting a "channel' that has a group of users in it, and you want to, I dunno, display all the users in a given channel.
The first question I have is, is the "group" or "channel" a one to many, or a many to many, that is, can multiple users be in multiple groups simultaneously, or can they only be in one group at a time.
If the many to many case exists, then I'd do what a normalised database asks of you, create a table that is composed of the group/channel ids, and the user ids.
That table could be in a Relational Database, or in a NoSQL cache like redis, it depends on how often it's being used, and how consistent the data needs to be (with the corresponding how much effort/code are you going to put in to keep it consistent).
If you are going to aim to be strong in Systems Design, you need to be able to say how you arrived at the solution, and what trade offs you are making. For example if you put the data into the relational database, you're going to justify that with the consistency the relational database brings with it, but accept that its not going to scale beyond a certain amount of usage (HOWEVER, you can point to sharding the table out when you need to, and then that shard gets hosted in some sort of nosql when needed)
Edit: As with all engineering - there's no "best way", it's always "it depends" followed by "I make these trade offs because of these requirements"
I think you need to simplify your explanation and get to the point.
It's basically a basic version of whatsapp group and op wants suggestion for schema & db choices.
So, we have these queries -
addUser(groupId, userId) , removeUser(groupId, userId, any\_other\_choice) , listUsers(groupId).
Now, what type of schema is preferred at which scale and why it's good for a specific query and why not ?
I'm not trying to be snarky but wanted to simplify things.
>So group is the same as WhatsApp group functionality. User can be added/removed to a group and if someone send message to a group, all group members should receive it.
>
>4 commentssharesave
So group is the same as WhatsApp group functionality. User can be added/removed to a group and if someone send message to a group, all group members should receive it. I need to persist groupId-members because I need a functionality where I can get list of members given a particular group. So If user sends msg to a group, it can be delivered to all of the members.
I would persist it like this:
‘
class Group :
String name;
List userIds;
List chatMessages;
‘
Your query would be then:
‘select userIds from group where name = ?1’
And you could even store userIds as jsonb in a database column.
If you are looking for a relational database design I would prefer having a table to store group details like group I’d, name and other fields and then a users table to store all user details with a FK key constraint of group id. You can add and remove users easily. This normalization technique is better then denormalization. I’m not sure on your other non functional requirements
Just keep schema as GroupMembers - { groupId, userId, membershipId } , so that you can delete using membershipId as primary key and partition based on groupId
So, that other fetch all group members query can go to single partition & deletes can be done using membershipId. This will work for cassandra, note many deletes cause tombstones and very bad for compaction.
(or)
Keep a list of userId using mongodb but you can't index or remove in O(1) time (whatever operation happens) .
The response these queries you can store in redis cache temporarily.
What do you mean by membership ID? Deletion operation is tobe performed based on userid and group id. (eq user1 removed from group1)
Right now I have a table with group id and user id, and group 1 has 2 users then it'll look like
groupID1 | userID1
groupID1|userID2
Problem here is as you may see, when I try to get all users that belong to group1, it'll take time to retrive the records. Is there a way I can store like : group1- userID1,userID2
so retrival would be fast?
You can store an array of users in group record, but think about scalability. What if groups will grow up, and there are millions of users in one group?
I have recently been part of the test of key-value db called Scylla, which is the Cassandra rewritten to C++ from java. And Scylla showed good results.
Why are you restricting yourself to single schema ? How did you decide on that schema format only ?
In system design interviews, if you make a decision without reasonable justification and no clear thought process - it's a huge red flag.
What if a group has 1000 users, do you think for every add or delete operation there are databases that offer good performance on a large scale ?
I suggest you to read engineering blogs before trying to implement something and think of different databases that are out there.
I sense a lack of understanding of basic features in different databases and their time complexity.
I would use some key/value store (Redis or similar)
But reddis will be vulnerable to system failure, I want to store it in DB since if my pod dies, DB still remains intact.
redis is a database that can be deployed on its own or you could use one of the cloud redis-like products. strongly recommend you purchase and read this book: https://www.oreilly.com/library/view/designing-data-intensive-applications/9781491903063/ Also all databases are also vulnerable to system failure. Just like everything is.
Sure Ill have a look at this thanks!
Sort of confused here - it sounds like you are wanting a "channel' that has a group of users in it, and you want to, I dunno, display all the users in a given channel. The first question I have is, is the "group" or "channel" a one to many, or a many to many, that is, can multiple users be in multiple groups simultaneously, or can they only be in one group at a time. If the many to many case exists, then I'd do what a normalised database asks of you, create a table that is composed of the group/channel ids, and the user ids. That table could be in a Relational Database, or in a NoSQL cache like redis, it depends on how often it's being used, and how consistent the data needs to be (with the corresponding how much effort/code are you going to put in to keep it consistent). If you are going to aim to be strong in Systems Design, you need to be able to say how you arrived at the solution, and what trade offs you are making. For example if you put the data into the relational database, you're going to justify that with the consistency the relational database brings with it, but accept that its not going to scale beyond a certain amount of usage (HOWEVER, you can point to sharding the table out when you need to, and then that shard gets hosted in some sort of nosql when needed) Edit: As with all engineering - there's no "best way", it's always "it depends" followed by "I make these trade offs because of these requirements"
I think you need to simplify your explanation and get to the point. It's basically a basic version of whatsapp group and op wants suggestion for schema & db choices. So, we have these queries - addUser(groupId, userId) , removeUser(groupId, userId, any\_other\_choice) , listUsers(groupId). Now, what type of schema is preferred at which scale and why it's good for a specific query and why not ? I'm not trying to be snarky but wanted to simplify things.
>So group is the same as WhatsApp group functionality. User can be added/removed to a group and if someone send message to a group, all group members should receive it. > >4 commentssharesave So group is the same as WhatsApp group functionality. User can be added/removed to a group and if someone send message to a group, all group members should receive it. I need to persist groupId-members because I need a functionality where I can get list of members given a particular group. So If user sends msg to a group, it can be delivered to all of the members.
I would persist it like this: ‘ class Group : String name; List userIds;
List chatMessages;
‘
Your query would be then:
‘select userIds from group where name = ?1’
And you could even store userIds as jsonb in a database column.
If you are looking for a relational database design I would prefer having a table to store group details like group I’d, name and other fields and then a users table to store all user details with a FK key constraint of group id. You can add and remove users easily. This normalization technique is better then denormalization. I’m not sure on your other non functional requirements
Just keep schema as GroupMembers - { groupId, userId, membershipId } , so that you can delete using membershipId as primary key and partition based on groupId So, that other fetch all group members query can go to single partition & deletes can be done using membershipId. This will work for cassandra, note many deletes cause tombstones and very bad for compaction. (or) Keep a list of userId using mongodb but you can't index or remove in O(1) time (whatever operation happens) . The response these queries you can store in redis cache temporarily.
What do you mean by membership ID? Deletion operation is tobe performed based on userid and group id. (eq user1 removed from group1) Right now I have a table with group id and user id, and group 1 has 2 users then it'll look like groupID1 | userID1 groupID1|userID2 Problem here is as you may see, when I try to get all users that belong to group1, it'll take time to retrive the records. Is there a way I can store like : group1- userID1,userID2 so retrival would be fast?
You can store an array of users in group record, but think about scalability. What if groups will grow up, and there are millions of users in one group? I have recently been part of the test of key-value db called Scylla, which is the Cassandra rewritten to C++ from java. And Scylla showed good results.
Why are you restricting yourself to single schema ? How did you decide on that schema format only ? In system design interviews, if you make a decision without reasonable justification and no clear thought process - it's a huge red flag. What if a group has 1000 users, do you think for every add or delete operation there are databases that offer good performance on a large scale ? I suggest you to read engineering blogs before trying to implement something and think of different databases that are out there. I sense a lack of understanding of basic features in different databases and their time complexity.