Dive Spotter

All the files of this project are saved in a GitHub repository.

Are you a scuba diver ?

No ?? Then, I highly recommend you becoming one ! Check out why.

Yes ? Then you know how difficult it can be to organize your next dive trip. There are so many criteria to consider to pick your next destination !

• The Type of Trip
  Are you a full-time diver ready for a liveaboard experience ? Or would you prefer to take breaks and visit the surroundings ?
• The Budget
  The cost of a dive trip can vary a lot depending on the destination: travel, accomodation, equipment rental, dive instructor, guide… Diving in the Maldives won’t impact your wallet the same way than diving in the UK !
• Your Level
  Are you a certified diver ? Are the local dive sites adapted to your level ? Are you ready to challenge yourself, or would you rather take it easy ? Strong currents, cold waters, low visibility… There are many parameters to consider to avoid surprises..!
• The Dive Sites
  Would you like to see corals ? Would you prefer experiencing The Blue ? Would you like to explore a wreck ? Each dive site offers a different experience, so pick wisely !
• The Fauna
  Are you interested in big animals, like whale sharks, manta rays and mola molas ? Or maybe you prefer tiny things like pygmy seahorse, pipefishes and mandarin fish ? You might have to plan your trip on the right period to maximize your chances to watch one of these…
• The Dive Shop
  Are the local dive shops well equiped ? Do they offer the certification and specialities you’re dreaming of ? Is your favorite instructor still working there ? Trustable dive buddies are essential for an enjoyable experience !

Dive Spotter is the recommendation engine which will help you to pick the dive trip of your dreams !

Concept

Divers have a strong community. The certification agencies, like PADI and SSI, encourage this sense of belonging through their training programmes and dive centers partnerships. Dive Spotter leverages this community to generate its advices and convey them to its users as if they were talking to their dive buddy.

Moreover, every diver is required to keep a logbook, which records the details of all their dives. Maximum depth, duration, dive site, weather conditions, technical settings, but also dive shop, instructor, dive buddy, encountered critters… These logbooks are requested at registration by dive centers, along with certification cards.

If a diver’s first logbook is usually hold in the form of a paper booklet, many divers quickly switch to a digital format, easier to travel with and allowing to attach the best snapshots of the day. Numerous solutions exist, but one online scuba logbook is particularly interesting: DiveBoard. This open-source platform is the largest online logbook on the market since 2011, gathering a huge dataset about scuba divers around the world: dive historics, dive site descriptions, dive centers reviews, critter pictures…

Dive Spotter is taking advantage of this data mine to provide divers with travel advices, recommend trustable dive shops, and make dive trips planning a walk in the park !

Libraries

The libraries used for this project include: recommenderlab for recommendation algorithms, ISOcodes to use standard country codes, dplyr for data manipulation, ggplot2, ggthemes and gridExtra for plotting, and kableExtra to display tables in HTML.

Demo Dataset

Source and Structure

DiveBoard is an open-source project, which can be reviewed on its GitHub account. The structure of the database is provided, as well as table contents (seed and seed data). Unfortunately, for understandable privacy reasons, the user data is only accessible through API queries.

As a consequence, the dataset used for this Minimum Viable Product (MVP) is only inspired from the information available in the DiveBoard database and its structure. It is built as the subset resulting from an Extract Transform Load (ETL) process using data related to dive records collected in the Maldives (dive spots, dive centers, users, critters…).

The dataset consists in three (3) tables:

• Divers Table
  Contains information on the divers, including their age, their certification level, the number of dives in their logbook, their cumulative depth, their cumulative bottom time, their longest dive, their deepest dive, their country of residence, as well as the list of dive types and critters they’d like to come across for their next dives.
• Sites Table
  Contains information on the dive sites, including their country, their region, their location name, their location coordinates, their name, their max and average depths, their average visibility, their average current, their average temperature on surface, their average temperature on bottom, their type, and the list of critters which can usually be spotted there.
• User Logbook Ratings
  Contains the rating from 1 to 5 for each dive sites the users already explored.

The dataset doesn’t contain any information about the dive shops, as this MVP will focus only on recommending dive sites. Further development will be necessary to provide additional recommendations to the users.

After importing the seed.sql file into MySQL, it is possible to get a list of dive sites in the Maldives.

SELECT  DiveBoard.spots.id,
        DiveBoard.spots.name,
        DiveBoard.spots.lat,
        DiveBoard.spots.long,
        DiveBoard.locations.name AS location_name,
        DiveBoard.regions.name AS region_name,
        DiveBoard.countries.cname AS country_name
FROM DiveBoard.spots
INNER JOIN DiveBoard.countries ON DiveBoard.spots.country_id = DiveBoard.countries.id
INNER JOIN DiveBoard.regions ON DiveBoard.spots.region_id = DiveBoard.regions.id
INNER JOIN DiveBoard.locations ON DiveBoard.spots.location_id = DiveBoard.locations.id
WHERE DiveBoard.countries.cname = 'Maldives'
AND UPPER(DiveBoard.locations.name) REGEXP 'ATOLL'
;

It is also possible to get a list of critters which have been spotted in the area.

SELECT  gbif.g_scientificnName,
        gbif.g_kingdom,
        gbif.g_phylum,
        gbif.g_class,
        gbif.g_order,
        gbif.g_family,
        gbif.g_genus,
        gbif.g_higherGeographyID,
        gbif.g_country,
        gbif.g_locality,
        gbif.g_habitat,
        COUNT(*) AS times_spotted
FROM DiveBoard.gbif_ipts AS gbif
WHERE gbif.g_country = 'Maldives'
AND UPPER(gbif.g_locality) REGEXP 'ATOLL'
GROUP BY gbif.g_scientificnName,
         gbif.g_kingdom,
         gbif.g_phylum,
         gbif.g_class,
         gbif.g_order,
         gbif.g_family,
         gbif.g_genus,
         gbif.g_higherGeographyID,
         gbif.g_country,
         gbif.g_locality,
         gbif.g_habitat
ORDER BY gbif.g_scientificnName ASC
;

Unfortunately, the common names list in English are incomplete, so only the scientific names are available. In order to simplify the understanding of this MVP, an arbitrary list of critters will be used, and the number of times they have been spotted on each site will be randomly generated.

Some parameters are set to generate the dataset tables.

set.seed(100)     #Seed
nDivers <- 1000   #Number of divers
target_user <- 2

#Arbitrary list of countries
sel_countries <- ISO_3166_1[ISO_3166_1$Name %in% c('France', 'United States', 'United Kingdom',
                                                   'Spain', 'Italy', 'Belgium', 'Norway', 'China',
                                                   'Maldives', 'Germany'),'Alpha_3']

#Arbitrary list of dive types
dive_types <- c('Sand', 'Coral Garden', 'Wall', 'Wreck', 'Pinnacles', 'Canyon')

#Arbitrary list of critters
critters_list <- c('Manta Ray', 'Jellyfish', 'Octopus', 'Whale Shark', 'Reef Shark',
                   'Mandarin Fish', 'Mola Mola')

Dive Sites Table

The dive sites are loaded from the CSV file exported from the DiveBoard database.

kable(head(sites, 10), caption = 'Dive Sites from DiveBoard') %>%
  kable_styling(bootstrap_options = "striped")

The table is then enriched with generated information:

• Maximum Depth: the maximum depth of the dive site (in meters).
• Average Depth: the average depth of the dive site (in meters).
• Visibility: the visibility of the dive site (in meters), which is how far the diver can usually see when underwater.
• Temperature at the Surface: the average temperature at the surface (in Celcius degrees).
• Temperature at the Bottom: the average temperature at the bottom (in Celcius degrees).
• Dive Types: the types of dive corresponding to the dive site. These booleans describe the topology and specific activities that can be found on the dive site.
• Critters: these booleans indicate which critters can be found on the dive site.

kable(head(sites, 10), caption = 'Dive Sites Table') %>%
  kable_styling(bootstrap_options = "striped") %>%
  scroll_box(width = "100%")

Divers Table

The divers table is generated with these attributes:

• Diver ID: the identifier of the diver.
• Age: the age of the diver.
• Certification Level: this integer indicates the certification level, from 1 to 4, 1 corresponding to the first certification level.
• Number of Dives: the number of dives performed by the diver.
• Maximum Depth: the personal depth record of the diver (meters). The maximum depth should be at least 10 meters.
• Maximum Bottom Time: the time of the longest dive of the diver (in minutes).
• Country of Residence: the code of the country of residence of the diver (alpha-3 ISO code).
• Cumulative Depth: the sum of maximum depths of all dives of the diver (in meters).
• Cumulative Bottom Time: the sum of time of all dives of the diver (in minutes).
• Dive Types Preferences: these booleans indicates the type of dives and activities the diver would like to perform.
• Critters Preferences: these booleans indicates the critters the diver would like to encounter.

kable(head(divers, 10), caption = 'Divers Table') %>%
  kable_styling(bootstrap_options = "striped") %>%
  scroll_box(width = "100%")

Logbooks Table

The logbooks table is generated considering the number of dives of each diver, with these attributes:

• Diver ID: the identifier of the diver.
• Dive Site ID: the identifier of the dive site.
• Rating: this integer indicates the rating given by the diver to the dive site, from 1 to 5, 1 corresponding to the worst score.

kable(head(logbooks, 10), caption = 'Logbooks Table') %>%
  kable_styling(bootstrap_options = "striped") %>%
  scroll_box(width = "100%")

Hybrid Recommenders

To provide users with the best recommendations, ones that are adapted to their level and preferences, it is useful to cluster them based on their number of dives. Experience is a big driver to recommend dive sites to a diver:

• Beginners (usually less than 20 dives, corresponding to the second certification level) would need to be hooked to the sport, so a safe exploration would be recommended.
• Confirmed (betweeen 20 and 100 dives) divers would want to be challenge, with a higher scope of dive sites - new adventures would be recommended to them.
• Pro divers (more than 100 dives) have already seen a lot, so their recommendations would be oriented to what they love, trying to make them discover new fields as well.

kable(head(beginners, 10), caption = 'Beginners Divers') %>%
  kable_styling(bootstrap_options = "striped") %>%
  scroll_box(width = "100%")

kable(head(confirmed, 10), caption = 'Confirmed Divers') %>%
  kable_styling(bootstrap_options = "striped") %>%
  scroll_box(width = "100%")

kable(head(pro, 10), caption = 'Pro Divers') %>%
  kable_styling(bootstrap_options = "striped") %>%
  scroll_box(width = "100%")

After preparing the data for modelling, the algorithms and parameters to be run for each user group need to be defined.

Beginners are at the start of their learning curve. These users would need to fall in love with the sport, explore, discover. But at the same time, novice divers have to deal with the apprehension inherent to scuba diving. Recommendations should be oriented to popular dive sites among their user group, with a slight random factor to push themn exploring the divers’ world and go beyond their comfort zone.

As a consequence, the selected algorithm for this user group is a hybrid recommender using this recipe:

• 50% Popular: the Popular algorithm recommends in priority dive sites which are popular among the beginner divers group. Beginners would tend to give a high score to dive sites in which they felt comfortable and discovered the underwater world.
• 30% Item-Based Collaborative Filtering: a dose of IBCF would help the user to slowly get out of their comfort zone, while staying in a relatively safe area. Beginner divers are more monitored by diving instructors, who would only recommend sites adapted to their level. Suggesting sites among other beginners’ logbooks seems a reasonable strategy.
• 20% Random: again, pushing the user to explore more and try new things would help to extend its scope. The Random algorithm will suggest dive sites which might set a new challenge to the user.

rec_beginners_1 <- Recommender(getData(split_beginners, "train"), "POPULAR")
rec_beginners_2 <- Recommender(getData(split_beginners, "train"), "IBCF")
rec_beginners_3 <- Recommender(getData(split_beginners, "train"), "RANDOM")
ensemble_beginners <- HybridRecommender(rec_beginners_1, rec_beginners_2, rec_beginners_3,
                                        weights = c(0.5, 0.3, 0.2))

Confirmed are divers who now feel confident underwater and would need to be encouraged to try new things. Safety is less a concern for these users, exploration is a must!

Thus, the selected algorithm for this user group is a hybrid recommender using this recipe:

• 30% Popular: on the same concept as for beginners, confirmed divers would tend to give a high score to dive sites in which they felt challenged or have seen new critters. Picking dive sites among their peers would help to suggest interesting opportunities for their level.
• 40% User-Based Collaborative Filtering: a higher dose of UBCF would help to find sites adapted to their level. These users represent the largest range of user levels, so the choices are numerous and should provide the users with satisfying recommendations.
• 30% Random: the Random algorithm provides once more new dive sites that might not be significantly present among this user group. It will help bringing new dive sites in the list.

rec_confirmed_1 <- Recommender(getData(split_confirmed, "train"), "POPULAR")
rec_confirmed_2 <- Recommender(getData(split_confirmed, "train"), "UBCF")
rec_confirmed_3 <- Recommender(getData(split_confirmed, "train"), "RANDOM")
ensemble_confirmed <- HybridRecommender(rec_confirmed_1, rec_confirmed_2, rec_confirmed_3,
                                        weights = c(0.3, 0.4, 0.3))

Pros are divers who might have experienced most of the possible udnerwater situations, and seen most of the critters in popular dive sites. These users would need recommendations that are fully aligned with their interests and preferences.

The selected algorithm for this user group is a hybrid recommender using this recipe:

• 40% Item-Based Collaborative Filtering: recommending dive sites which are popular among their elite group seems reasonable, as easy dives might not satisfy these users’ thirll for challenge and adventure.
• 30% Rerecommender: professional divers know what they love. And they would repeat it again and again if it’s to avoid boring dives. The Rerecommender algorithm will pick dives that they really enjoyed, for them to appreciate every minute spent underwater.
• 30% Random: again, the Random algorithm will help these experienced divers to discover new things, beyond their usual playground.

rec_pro_1 <- Recommender(getData(split_pro, "train"), "IBCF")
rec_pro_2 <- Recommender(getData(split_pro, "train"), "RERECOMMEND")
rec_pro_3 <- Recommender(getData(split_pro, "train"), "RANDOM")
ensemble_pro <- HybridRecommender(rec_pro_1, rec_pro_2, rec_pro_3,
                                  weights = c(0.4, 0.3, 0.3))

These hybrid models being defined, it is possible to provide recommendations to users from each of these groups. The tables below give some examples, based on the logbooks of a user of each user group.

kable(head(top_15_ensemble_beginner, 10), caption = paste('Recommendations for Beginner Diver', target_user)) %>%
  kable_styling(bootstrap_options = "striped") %>%
  scroll_box(width = "100%")

kable(head(top_15_ensemble_confirmed, 10), caption = paste('Recommendations for Confirmed Diver', target_user)) %>%
  kable_styling(bootstrap_options = "striped") %>%
  scroll_box(width = "100%")

kable(head(top_15_ensemble_pro, 10), caption = paste('Recommendations for Pro Diver', target_user)) %>%
  kable_styling(bootstrap_options = "striped") %>%
  scroll_box(width = "100%")

Content-Based Recommender

As each user maintains a mandatory logbook, in which it lists its appreciation for eadch site, and as information about dive sites is available and reasonably informed, it is possible to use a Content-Based algorithm to provide recommendations to the user, without considering other users’ information. The recommendations provided will thus be more personalized, and might lead the user to dive sites which are not necessarily well known by its user group.

Using a linear regression, based on the user’s logbook and the entire dive site list, a new list of recommendations can be provided to the diver.

Recommendations Visualization

The dive site recommendations can be visualized on a map-style scatterplot, to identify if there is any specific area in the Maldives more adapted to specific group of users. Comparing the results to the complete map of dive sites gives a good outlook of the dive sites landscape in the region.

Conclusion

This Recommendation System MVP provides a good idea of what could be achievable with Dive Spotter regarding dive sites recommendations. The model could be extended to dive shops and provided to travel agencies, enabling a promising revenue stream if deployed on a large scale.