Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add additional operations such as List/Set operations #80

Closed
tlvenn opened this issue Sep 22, 2016 · 9 comments · Fixed by #87
Closed

Add additional operations such as List/Set operations #80

tlvenn opened this issue Sep 22, 2016 · 9 comments · Fixed by #87
Assignees
Milestone

Comments

@tlvenn
Copy link

tlvenn commented Sep 22, 2016

Not that is a bad thing by itself at all but i am wondering if there is any plan to support List and Set operations in the future ?

I believe it would open up many more use cases for cachex which arguably would go beyond just caching..

@whitfin
Copy link
Owner

whitfin commented Sep 22, 2016

@tlvenn do you mean similar to the commands Redis offers?

The issue here is that we back a cache with an ETS table, which only supports get/set by default. I can add things like list operations but it would have to be known that they're going to be a transactional get/set under the hood. Unless we move away from ETS (which might be viable now distribution is gone), there's no better way to implement that.

Are you interested in the features because of the sugar (i.e. convenience)? Or do you want O(1) perf against them, etc?

@tlvenn
Copy link
Author

tlvenn commented Sep 22, 2016

Does not have to match exactly what redis offer but you get the idea. ETS table are indeed problematic and while convenience would already be very nice, i have the feeling people will just assume it can perform on the same complexity level as Redis and it will not be pretty ;)

I guess I am just wondering what the roadmap is and if such thing is outside of the scope of cachex.

@whitfin
Copy link
Owner

whitfin commented Sep 22, 2016

@tlvenn so the scope is just whatever is requested, as long as it doesn't damage what's already there ;) A Set op would be pretty fast still, probably 10µs per operation (for inserts), so it's not horrible

@tlvenn
Copy link
Author

tlvenn commented Sep 22, 2016

Good to hear @zackehh , then I honestly think it would be nice to have basic set and list semantics.

@whitfin whitfin changed the title Plain key-value store only ? Add additional operations such as List/Set operations Sep 22, 2016
@whitfin whitfin self-assigned this Sep 22, 2016
@whitfin
Copy link
Owner

whitfin commented Sep 22, 2016

@tlvenn just so you're aware, you can use get_and_update/4 in the meantime to have those types of semantics. You'd have to handwrite the modifications made but you could easily do so, and it would still be sufficiently fast.

@tlvenn
Copy link
Author

tlvenn commented Sep 23, 2016

Ya i was thinking it would be best to keep data manipulation as close as the data source itself from a process perspective.

@whitfin
Copy link
Owner

whitfin commented Sep 23, 2016

@tlvenn I'm thinking about creating this type of interface internally:

lpop = fn([ head | tail ]) ->
  { head, tail }
end

rpop = fn
  (list) when length(list) > 0 ->
    { List.last(list), :lists.droplast(list) }
  (list) ->
    { nil, list }
end

Cachex.start_link(:test, [
  commands: [
    last: { :return, &List.last/1 },
    lpop: { :modify, lpop },
    rpop: { :modify, rpop }
  ]
])

{ :ok, true } = Cachex.set(:test, "my_list", [ 1, 2, 3, 4, 5 ])

5 = Cachex.invoke(:test, :last, "my_list")
1 = Cachex.invoke(:test, :lpop, "my_list")
2 = Cachex.invoke(:test, :lpop, "my_list")
5 = Cachex.invoke(:test, :rpop, "my_list")
4 = Cachex.invoke(:test, :rpop, "my_list")
3 = Cachex.invoke(:test, :last, "my_list")

[3] = Cachex.get!(:test, "my_list")

That way you can define custom functions on the cache (from outside). Internally we can just use this to surface some of the more common functions, perhaps those which can be optimized. Any thoughts?

Trying to make this extensible, rather than Cachex having to implement lots of different use cases.

@alexgleason
Copy link

Hi, I'm thinking about using Cachex to pre-generate a feed of posts (on a per-user basis) for a Twitter-like application.

The thing I'm concerned with is race conditions. If I call get_and_update/4 10 times per second on the same key (to add elements to the list), do you think it would be a problem?

@whitfin
Copy link
Owner

whitfin commented Jan 2, 2024

@alexgleason this got lost in my mailbox, so I'm waaaay too late to notice this, but for anyone coming back to this thread: no, calling it multiple times on the same key will run them sequentially and avoid races.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants