• Welcome to ScubaBoard

  1. Welcome to ScubaBoard, the world's largest scuba diving community. Registration is not required to read the forums, but we encourage you to join. Joining has its benefits and enables you to participate in the discussions.

    Benefits of registering include

    • Ability to post and comment on topics and discussions.
    • A Free photo gallery to share your dive photos with the world.
    • You can make this box go away

    Joining is quick and easy. Login or Register now by clicking on the button

Assistance in Gathering Diver Log Databases for Research Purposes

Discussion in 'Research and Development' started by jnrpal, Oct 22, 2020.

  1. jnrpal

    jnrpal Garibaldi

    # of Dives: 500 - 999
    Location: Australia

    Looking for some assistance in gathering a collection of dive logs to do some statistical data analysis in conjunction with the DAN published annual reports.

    Is anyone aware of any of the available dive log software programs/apps having the ability to export out mass amounts of data of dive logs of multiple users input?

    Looking to build a mass database of dive profiles to analyse the data, the more data the better. I know DAN was aiming to have 1,000,000 collected dive logs from years and years ago, but a lot of links from previous discussions related to this are no longer hosted.

    Appreciate any assistance with the above or any contacts that you may believe could be a good starting point.

    Kind Regards,
  2. stepfen

    stepfen ScubaBoard Supporter ScubaBoard Supporter

    # of Dives: 200 - 499
    Location: Greece
    Hi and welcome to ScubaBoard.

    I can't really understand how you plan to do this.

    You have a very serious bias straight from the beginning if you limit yourself only to digital data because you inevitably exclude less technology oriented divers (for example older ones) or very new divers (they tend to dive without dive computers at the beginning) from your sample.

    Apart from this.Where do you plan to get your data from??? There are two options:
    A. Ask divers to voluntarily donate/contribute their dive logs to you, or
    B. Ask dive log companies (I hope @dirkhh from subsurface doesn't see this - he hates calling subsurface a company and it is not a company but I use this term so that the OP understands what we are talking about) to donate/contribute their users' data.

    Both approaches have very serious limitations.
    In particular in case A since people will participate voluntarily you will get biased results. A diver who have some f*cked up dives won't be willing to share, and so on. Also how do you let divers know about your research?? If you post it in places like here in SB you will get very biased responds. In here most of divers have 1000s of dives and years of experience, usually have 10s of tanks/regs/BPW/etc, almost all (except maybe @Angelo Farina :) ) use frog kick, "never" break any rule (or any rule they dim important :) ) and so on. I am exaggerating a bit but it is a known fact that active users here in SB are usually much above the "average" diver.

    In case B, in principle you should be able to get better (less biased) results BUT I would think there are serious legal considerations. I would guess that no such company or organization like subsurface will be willing to share with anyone their user's data. And for sure most users won't be happy to know that their data can be used by outsiders.

    I would be concerned first on how to address the above. How to do the actual data gathering is something you will have to figure out once you have settled these first.

    As always that's just my 2c - I might be totally wrong,

    All the best
    Angelo Farina likes this.
  3. atdotde

    atdotde Solo Diver

    # of Dives: 200 - 499
    Location: Munich, Germany
    As far as Subsurface is concerned: Nobody except the user looks at the data stored in the Subsurface cloud, this is a promise of privacy to the users. Not even the Subsurface team. The only exception is that only after given explicit permission, @dirkhh will look at a user's data when trying to debug a problem.

    So no, this treasure trove of dive data is not available for any kind of research (if it were, I would have run some statistics on it long ago).

    Some time ago, I had tried to collect log file donations from users on my web page but that got nowhere. This even with Subsurface having the possibility to export the log with all text data removed ("anonymise").
  4. jnrpal

    jnrpal Garibaldi

    # of Dives: 500 - 999
    Location: Australia
    Genuinely appreciate the input atdotde and stepfen,

    Completely understand the biased data collection from a place like this, which is something that will be taken into account once initial testing of data analysis is completed and confirmed to be of efficient utilisation. Essentially was just looking for a base data log to test the software on for initial feasibility of analysis to be assured, not looking to impeded on anyone's privacy or anything of the sort. I can utilise my own logs as a baseline at a minimum and once feasibility of analysis is assured then will be in further contact with DAN with an in depth explanation of the utilisation of the data.
    Tad hand tied with the extent in which I can explain the application, unfortunately due to the early stage of this specific use case of the implementation of the data analysis software, but very exciting tech that I genuinely hope makes some inroads into the safety of divers from recreational through to commercial.

    However, was just seeing if there was any open source data available (non id'd and user information registered as simply diver 0001 for e.g.) but understand both points made, and as soon as we have a bit more publicly available information will update this post and will give some clarity as to what and why the information was trying to be sourced.

    Will just proceed with the medical approach, and through those channels, was just looking to save some time and give more variance of data than just my own logs, to be able to perform a small scale run-up and analysis before proceeding with the full analysis (a far more costly exercise). This initial data request wasn't for the end goal utilisation, more of a test run of the integration.

    Sorry if this sounds murky in any way shape or form, that is the furthest from the team's and I's intention. Hope this gives a better explanation and happy to answer any further questions in the meantime or just for a general scuba chat haha.
  5. Edward3c

    Edward3c Instructor, Scuba

    With more countries / states (e.g. Egypt, Thailand, California) introducing personal data protection legislation in line with Europe’s GDPR your going to have difficulty in finding any organisation willing to help.
  6. Sifossifoco

    Sifossifoco Nassau Grouper

    # of Dives: I just don't log dives
    Location: Private
    I am not sure I understand this. Would divers have to provide logs without even knowing the supposed use? That's quite a big ask.
    Curious to know what the 'medical approach' involves. You didn't actually say this was going to be a medical application, and if this is the case I am pretty sure any data harvested from ordinary logs would not be adequate. It sounds like you are in some sort of big data project, and we may or may not be able to help, but something more should be disclosed.
  7. WeRtheOcean

    WeRtheOcean Barracuda

    # of Dives: None - Not Certified
  8. scubadada

    scubadada Diver Staff Member ScubaBoard Supporter

    # of Dives: 1,000 - 2,499
    Location: Philadelphia and Boynton Beach
    Whatever happened to DAN's Project Dive Exploration. I contributed dives to that for many years. I have never seen the output.

    If DAN can't get it done, how do you expect to?

Share This Page