Field to Database
| Data Carpentry | |
|---|---|
| Quick Links for Data Carpentry | |
| Link to Agenda | |
| Workshop Presentation Biblio Entries | |
| Workshop Blog | |
This wiki supports the Workshop - Field to Database: Biodiversity Informatics and Data Management Skills for Specimen Based Research. Where? The University of Florida at iDigBio from March 9 - 12, 2015. It is the third in a series of four biodiversity informatics workshops planned in the upcoming year (2014-2015) for iDigBio. The fourth workshop in this series is Sept 15-16, 2015 and focuses on Data Management for Collection Managers.
General Information
This workshop's aim is to investigate current trends in collecting, and focus on best practices and skills development for supporting the collection of robust, fit-for-research-use data. This 4-day short course is designed to be hands-on and will mix lectures with field work and participant exercises and presentations.
Our curriculum includes:
- (synopsis here)
The concepts, skills, and tools we teach are domain-independent, but example problem cases and datasets will be taken from organismal and evolutionary biology, biodiversity science, ecology, and environmental science.
Skill Level: Some exposure to R is recommended. This workshop is not explicitly for beginners. But R experience is not mandatory. If you are new to R, we recommend an intro to R coursera course prior to the workshop.
Updates will be posted to this website as they become available.
Software Installation Requirements and additional information is available at the github web site for the project: <github links to datasets here>
Planning Team
François Michonneau (FLMNH - iDigBio), Katja Seltmann (TTD-TCN, MNH), Pam Soltis (FLMNH - iDigBio PI), Derek Masaki (USGS - BISON), Deb Paul (iDigBio), Shari Ellis (iDigBio), Kevin Love (iDigBio)
About
Instructors: François Michonneau (FLMNH - iDigBio), Katja Seltmann (TTD-TCN, AMNH), Derek Masaki (USGS)
Assistants: Dan Stoner (ACIS - iDigBio), Deborah Paul (FSU - iDigBio), Matt Collins (ACIS - iDigBio)
Who: The course is aimed at graduate students, postdocs, research staff, and other researchers.
Skill Level: Non-beginners with respect to R. Some exposure to R would is recommmended. This workshop is not explicitly for beginners. But R experience is not mandatory. If you are new to R, we recommend an intro to R coursera course prior to the workshop.
Where: iDigBio in Gainesville, FL
Requirements: Participants must bring a laptop with a few specific software packages installed. If you will be traveling from out of town, you will need to make your own travel arrangements.
Contact: Please email Deb Paul, dpaul@fsu.edu for questions and information not covered here.
Twitter: hashtags
Tuition for the course is free, but prior registration is required for attending. You can register here.
Workshop Evaluation
- link to pre-workshop survey (if we do one)
- Post Workshop Survey Results
Software Installation Details
Software needed for Data Carpentry Workshop at iDigBio
- link to software installation instructions
- You must RSVP that the required software is installed, prior to the workshop. Instructors are available to help - see your email for their contact information.
We use Adobe Connect extensively in this workshop. Please perform the systems test using the link below. Also, you will also need to install the Adobe Connect Add-In to participate in the workshop.
Agenda
- AdobeConnect #field2database Room (if we are going to use).
- Pre-workshop meeting and dinner at Piesano's, 6 PM Sunday March 8th, 2015. Piesano's is at NW 13th St. and 1250 W. University Ave. in Gainesville. All are welcome. Please do RSVP to Deb Paul, dpaul@fsu.edu
| Course Overview - Day 1 | ||
|---|---|---|
| 8:30-8:45 | Introductions & Overview, Data Carpentry: Making data science more efficient | All, Deb Paul |
| 8:45-9:00 | Linking Heterogeneous Data in Biodiversity Studies: the need for data carpentry | Pam Soltis, iDigBio PI |
| 9:00-10:00 | Better use of spreadsheets, part I | Tracy Teal |
| 10:00-10:30 | Break | |
| 10:30-12:00 | Better use of spreadsheets part II | Tracy Teal |
| 12:00-1:30 | Lunch (with OpenRefine Demo) | Deb Paul |
| 1:30-3:00 | SQL Introduction | Matt Collins |
| 3:00-3:30 | Break | |
| 3:30-5:00 | SQL part II | Matt Collins |
| 5:00-5:30 | Review / Wrap up for tomorrow | |
| Course Overview - Day 2 | ||
| 8:30-10:00 | Introduction to the shell | Tracy Teal |
| 10:00-10:30 | Break | |
| 10:30-12:00 | Introduction to R | François Michonneau |
| 12:00-1:30 | Lunch | |
| 1:30-3:00 | Manipulating and plotting data in R | François Michonneau |
| 3:00-3:30 | Break | |
| 3:30-4:30 | Getting data in and out of R: How to integrate R in your workflow | François Michonneau |
| 4:30-5:00 | Sharing your data and your results: RMarkdown and Figshare | François Michonneau |
| 5:00-5:30 | Review / Wrap up / Evaluation and Feedback | |
Future plan: Scaling it up: Demo using the iPlant Discovery Environment (DE)
Link to Workshop Report
Logistics
- Logistics & Hotel Information (for the out-of-towners)
- Where to find food
- Workshop Calendar Announcement
- Participant List
Adobe Connect Access
Adobe Connect will be used to provide access for a remote classroom at the American Museum of Natural History. Workshop participants will be encouraged to be logged in to the Adobe Connect room to facilitate sharing with this remote group: Already registered and accepted?
- Adobe Connect Room for Data Carpentry at iDigBio
- AMNH (remote) and iDigBio participants take notes together with this EtherPad
Presentation Documents and Links
Links to any presentations (like power points) here.
- Data Carpentry: Making data science more efficient Presenter: Deb Paul, iDigBio
- Linking Heterogeneous Data in Biodiversity Studies: the need for data carpentry Presenter: Pam Soltis, iDigBio PI
- GitHub repository for Data Carpentry Workshop at iDigBio
- EtherPad for Data Carpentry at iDigBio: workshop notes
- SQL commands list
- SHELL commands list
- R lesson
- Getting started with Open Refine
Workshop Recordings
Day 1
- 8:30am-10:00am http://idigbio.adobeconnect.com/p7gqmvndl5a/
- 10:30am-12:00pm http://idigbio.adobeconnect.com/p6ipl39xnl3/
- 12-12:30 http://idigbio.adobeconnect.com/p4dsgo0y77y/
- 1:00pm-5:00pm http://idigbio.adobeconnect.com/p4rm50ukzv5/
Day2
- 8:30am-10:00am http://idigbio.adobeconnect.com/p2crpl68wr4/
- 10:30am-12:00pm http://idigbio.adobeconnect.com/p1e150mot75/
- 1:30pm-5:00pm http://idigbio.adobeconnect.com/p4t16mcnvvk/
Data Carpentry Resources and Links
- Data Carpentry Materials on GitHub
- Ten Simple Rules for the Care and Feeding of Scientific Data. Goodman et al
- Code and Data for the Social Sciences: A Practitioner's Guide. Matthew Gentzkow, Jesse M. Shapiro Chicago Booth and NBER March 10,2014
- Nine simple ways to make it easier to (re)use your data. White et al.
- You want to learn SQL independently? Try Head First SQL
- Head First Excel, O'Reilly
- Check out DataONE
- They've got a great Software Tools Catalog
- Put standard metatdata with your data. Wondering how to do that? Check out DataONE's Morpho Tool available under the tools menu at https://knb.ecoinformatics.org/.
- Why? Makes your data re-useable, and better still, makes your data discoverable. Get cited for your datasets in addition to your published papers!
- Using Open Refine? Want to compare your taxon names against a standard list? Try this reconciliation service.
- Read Gaurav ' Blog post first: http://gbif.blogspot.com/2013/07/validating-scientific-names-with.html
- Then, give it a try. The google plus Open Refine community will help you figure it out (it's not hard).
Links from You
- How about you? Got a favorite resource - a book?, a website? to share with your classmates?
- Data Science at the Command Line
- Free Training Resources for UF students, faculty, and staff UF provides free access to over 2600 online training courses through Lynda.com. Does your institution have similar free training opportunities?
Related Blog Posts and Photos
- Inaugural Data Carpentry Workshop by Tracy K. Teal
- Our First Data Carpentry Workshop by Karen Cranston
- Tales from the First Data Carpentry Workshop by Deb Paul, May 2014
- Data Carpentry, Please can we have some more?! by Deb Paul, 15 Oct 2014
- Data Carpentry Facebook Photo Album