If I had the need to keep a (potentially) large list (10k - 100k) of relatively short (300 characters-ish) strings what's the best way to do that in a plugin? Is there a "standard"/"best practices" way?
Text file -- slow and disk intensive
DB -- but where/how?
BerkleyDB -- more complexity
Memcache -- even more complexity
Thanks!
Best way to keep a dynamic (large) list of "stuff"
Re: Best way to keep a dynamic (large) list of "stuff"
BTW -- I'm planning to implement a couple of these, but what should be canonical?
-
- Core Developer
- Posts: 30022
- Joined: Tue Sep 16, 2003 9:45 pm
- Location: Cologne, Germany
- Contact:
Re: Best way to keep a dynamic (large) list of "stuff"
Hi!
Good point. I think if you need lookup features, i.e. fetch certain strings depending on conditions, the DB is the only proper way to deal with. Make the plugin create its own database (see staticpag eplugin, look for "setup" method and how its called, andhow it uses "CREATE TABLE..." sql strings.
BerkeleyDB and memcache have too many dependencies so they should be avoided by default and maybe only added as an option.
I would use a textfile, if you always load the full set of data/strings though. This is faster than passing data from the SQL server to your application through a socket.
Regards,
Garvin
Good point. I think if you need lookup features, i.e. fetch certain strings depending on conditions, the DB is the only proper way to deal with. Make the plugin create its own database (see staticpag eplugin, look for "setup" method and how its called, andhow it uses "CREATE TABLE..." sql strings.
BerkeleyDB and memcache have too many dependencies so they should be avoided by default and maybe only added as an option.
I would use a textfile, if you always load the full set of data/strings though. This is faster than passing data from the SQL server to your application through a socket.
Regards,
Garvin
# Garvin Hicking (s9y Developer)
# Did I help you? Consider making me happy: http://wishes.garv.in/
# or use my PayPal account "paypal {at} supergarv (dot) de"
# My "other" hobby: http://flickr.garv.in/
# Did I help you? Consider making me happy: http://wishes.garv.in/
# or use my PayPal account "paypal {at} supergarv (dot) de"
# My "other" hobby: http://flickr.garv.in/
Re: Best way to keep a dynamic (large) list of "stuff"
Yeah -- I'm thinking at some point in the future I will want to query the list one way or another. I'll shoot for the DB. Thanks!