I think it is time to start on the SQL backend. Would you like to help? --Dsblank 11:46, 24 March 2009 (UTC)
Yes I would very much. If fact it's exactly why I'm trying to join the project. The sql discussion page is solely responsible for my renewed interest in gramps. I'm just starting to get involved though. I was going to join the mailing lists today. I still need to read the dev pages and explore a more dev friendly install of gramps on my pc. I also need to familiarize myself with the code. Also I haven't told anyone yet but I've started work on a windows binary using py2exe. So far it compiles and I can edit a user. I still need to do a few more things to it before I can release an alpha of it though. I was hopeing to get out an alpha of that (few days to a week) before I switched focus to the sql backend. but thought I'd start getting involved in the sql discussion as I know that can take a bit with open source projects. I had a couple of questions but I'll post them in more appropriate places. --AaronS 19:57, 24 March 2009 (UTC)
Yes, I saw your questions... basically, you are at the beginning. There are only a couple of points, but I think you hit them: pick a DBI; decide a table structure. If you focus on the first, I'd be glad to bring the schema to at least a working point. --Dsblank 22:04, 24 March 2009 (UTC)
Sounds good I'll try to do some reasearch into DBI's and post it on GEPS 010: SQL Backend. I'm new to the project and am not yet familiar with it's politics yet. what is the other developers level of support for this change? Is this a supported direction or a yea sure maybe. or even worse a fork. --AaronS 22:45, 24 March 2009 (UTC)
Well, it is a sensitive subject, but if we do it within the defined GRAMPS DB interface, there shouldn't be any issue. I think that the sensitivity comes from the fact that the project "just" switched from XML to BSDDB. This happened a couple of years ago now, but I think some don't want to think about this again. However, there are so many issues in the bugtracker that it is hard to justify keeping BSDDB. And, I think it will become faster and easier to develop with SQL. I want to develop some GRAMPS webapps, too, so I see this step as critical on that front, but is still independent of that. One of the GRAMPS leads (Benny) has advocated looking at Django, so he is interested in doing it correctly. So, yes, I think that this has some support, although it may be a small group initially. I have no intentions of a fork. --Dsblank 00:10, 25 March 2009 (UTC)
from xml to bsddb? on the surface that sounds like a strange switch but I don't know anything about bsddb other than it's a flat file and not relational. good good about the forking. while there do come times to do them forks are the devil. That's so funny he was looking at Django. My looking at building something with Django is what eventually led me back to gramps! I guess that I should go ahead and explain my position and where I'm coming from. In the last few years I've started gaining a stronger interest in genealogy. I inherited a few gedcoms and misc dbs from family members. being a software developer myself I had a keen interest in the software used. I started noticing a few major problems with the current data storage solutions. long story short my main business requirements boiled down to:
- a relational db with a table schema that made sense. after all that data entry I wanted my ad hoc queries.
- hopefully using technologies that would still be somewhat relevant in 5 to 10 years. ie not using tech already waning.
After a long search including gramps I just couldn't find what I wanted. So I started writing my own software. I used a The Uniform Server to provide a run any where WAMP. ie Apache, MySQL, and PHP. The thought was that even though I was going to run it locally by using the basic web techs the software would be highly portable and hopefully stand the time test OKish. I used ATK (A php framework for quickly developing websites with a business application focus) for the front end and business logic. I did get the db designed and the ui complete and most of the business logic done as well. I was just about to start working on gedcom support and reports (necessary before even an alpha release) when the project got shelved for a while. It turns out that ATK was the wrong choice. ATK isn't as mainstream as some of the others but it's more relevant focus seemed to be a good match. Howerver, as I've been developing another project using it I've discovered that while it's beautiful for quick fairly standard crud with standard table relationships it was almost always a bear every time I needed to do something that didn't come packaged. Which is why I was shopping around again now that I've got around to working on my genealogical data solution again. Since I got burned with ATK i definitely wanted a much more mainstream framework with good documentation and a strong user base. Django kept popping up and I decided to give that a try perhaps using a tech stack something like this which lead me to search for what kind of python gedcom libraries there might be which lead me back to gramps and GEPS 010: SQL Backend so I decided to try the windows install steps again and since they worked this time (some one has done some work since initial testing) I decided to invest some time to see if I can make gramps meet my requirements. Lord knows I certainly don't want to develop my own project if I can help it. --AaronS 06:01, 25 March 2009 (UTC)
The history behind the move from XML to BSDDB is that GRAMPS version 1 originally used a compressed XML file as it's permanent storage. The entire family tree was read from the XML file, held and manipulated in memory, then written back to the file when finished. This problem with this approach is that it didn't scale well to larger family tree sizes, so, version 2 included BSDDB as a backend to provide storage. This helps performance because the entire data set did not need to be read/written in one go. XML is still there as an export option and is the recommended approach for archiving your database. I think any 'sensitivity' to introducing any form of SQL or RDBMS backend is not to do with the move to BSDDB but more with where it fits in the order of priorities. We have a limited number of developers and each have different interests they want to pursue. I personally would like to see BSDDB replaced with a database API or ORM which ultimately uses SQL to talk to a variety of backends. We could then offer SQLLite as a low maintenance like-for-like replacement of BSDDB and the opportunity to use a proper RDBMS for those with the technical ability to support it. I don't expect to see a SQL backend replacing BSDDB in the next version, but be aware that there is a 3.2_Roadmap page on the wiki for the next version and you can see there the things I am likely to be concentrating on. --Gburto01 14:47, 25 March 2009 (UTC)