The Field Research Study

Music Journalist

The Field Research Study

While working with the developer relations program at Facebook, one of my data science initiatves involved understanding and improving meaningful engagement in online groups. I have written about meaningful engagement, so if you are curious about what it is please read that post. Facebook groups bring together members with similar interests, but for developers they are online spaces where engineers share technical resources. As a former software engineer living in the Bay Area, California (tech-hub), there were many in-person events that I could attend where I could meet and engage with other technical folks and trouble shoot the bugs in my code. I was unaware, at the time, of how developers in the developing world would troubleshoot so I spent alot of time reading posts from active groups (groups who were posting often) to understand the subject area of the posts.

There were 119 developer circle groups from all over the world who held in person events under the leadership of volunteer group members, whom we called Dev-C leads. The Dev-C leads would communicate directly with a regional program manager about the specs of the events (date, time, # devs expected), and the program manager could talk generally about the happenings in the region they were in charge of. The EMEA program manager. mentioned over breakfast that there was a very active female-lead group in North Africa that we should support in person. Our team at the time was very small and everyone had a lot on their plate so creating the time to go to the in-person event in North Africa was a huge commitment.

I started to pay close attention to the online group in Tunis, Tunisa (North Africa) that the program manager mentioned and got to know the Dev-C lead over messenger chat. She informed me that they were having a deep learning hackathon and asked if I could support. I was so excited by the offer and I immediately spoke to my manager about it. My manager said that if I created a project plan and demonstrated in the plan how I could use the trip to improve the meaningful engagement model, that I could go.

This is the story of how I created the project plan, the tools I used to create one and how I improved meaningful engagement in one online group.

Jan Chipchase and The Field Study Masterclass

What was the class?

Jan Chipchase is an international user experience researcher who is well-known for going into the developing world and helping large organizations establis a presence there by understanding the local customs and traditional norms and explaining that to the development team of the large company. A great example of this is McDonalds building it’s first location in India. There are certain parts of India where it is disrespectful to eat meat and so McDonalds had to think about how to change the menu to fit the new customer segment.

What did I learn?

I learned how to listen and build trust with people living in the developing world given the high potential for fraud. Online spaces have made it a bit easier to understand what is going on and where and more news segments like Vice, and Vox are covering the political happenings in parts of the developing world, but if you are planning a trip you need to make sure that you are safe. I learned how to use the internet to find a credible palce, this could be a hotel and ask the hotel management for help finding a guide. A guide is a local in that community who lives there and speaks your language. This guide needs to understand your goals and what it is you would like to see so they can plan accordingly and understand what to show you. It is important for you to pay this guide, obviously, but it might even make some sense to pay using a task management system, so you can say once you coplete this tak I will give you this much money and then continue to follow through.

How did I apply what I learned in my project plan?

Initialy I used the Dev-C lead as my guide and I was able to get a lot of resources from her, this is until I found a better guide. I found Yahya, and learned all about the developer ecosystem from him. I even interviewed him and was able to travel to some remote places that I would not have been able to otherwise.

Cultural Context and Finding a Guide

I used the online group to connect with the Dev-C lead who lived and worked as a software engineer in Tunis, Tunisia. Once I made a decision to actually travel to Tunisa, there were so many parts of the trip that I needed to plan. I needed to:

I can write a bit more about the international travel and all of the internal project planning that went into making this happen but the most important bit of information here concerns the hotel, which was recommended by the Dev-C lead.

I would also like to make a note here around electricity, baggage and customs.

International adapters are needed to plug in products made for and in America. The wall outlets in other countries are different and you may blow a fuse if you try to just plug in your product or device into the wall. I would suggest going to Best Buy or ordering an international wall outlet online. Here is an image of the outlet that I used.

Packing for an international business trip can be overwhelming the first time, but pro-tip, just pack one bag and ensure that you have maybe two main outfits and undergarmets. Maybe one dress. Wear something comfortable to and from the airport and shoes that are going to work for the weather where you are traveling.

Your things might get lost if you check your bag but if you need technology like a computer or something else ensure you get it in the USA or UK because it WILL NOT be available once you land in the developing world.

Before you leave the airport where you land use the ATM to pull out cash, you don’t know where you are goint to come across another ATM and it could be dangerous to pull out cash on the street. Finding a bank isn’t the easiest task and if you don’t have your SIM card situation figured out with cellular service, this could be a problem. Having cash in the local currency allows you to call a cab and give them the name of the airport or just grab food if need be.

Once I landed in North Africa, Tunis, and checked into my hotel I tok a stroll around the neighborhood. Because my orginal guide seleced the hotel and it was 5 stars I knew it was an okay neighborhood. Walking around I found signs for a developer community and I walked in, this is where I found another guide and he took me all around Tunisia.

The Deep Learning Hackathon

What was it?

Where was it?

Cogite office space, a co-working space in Tunis, Tunisia.

Introducing reinforcement learning

Pieter Abbeel is a robotics instructor at UC Berkeley and I live and study in Berkeley so I am familiar with his work. The computer science research community is very small and most of the reserachers aggregate to particular institutions and not very many of these reserachers are in EMEA of North Africa. I wanted to ensure the students had access to the highest quality up to date information so I reached out to Pieter and grabbed online material which I could share.

What did I learn about the developer ecosystem that I did not know before?

I learned that developers in emerging countries would like to be up to date on current trends and build things for America. This is very strane because there are so many opportunities for them to build things that would serve the population of people where they live. For instance I learned that in North Africa all of the higher education is free but the courses are not world class and the infrastructure for the records and information mangement systems is awful or null and void. I think there is a lot of privilege in terms of being a student in America.

I also took away the fact that theory and application are two different things, so it’s easy to work through a course online and work throught hte projects but in the real world and in the working world you need to understand business priorities and why the business is making certain decisions or not and where your job plays into that. I think that more students from the developing world should take google certification courses and use those skills to enter jobs in technology. Work is turning remote and many of the people who would like to traditionally go to college only need to work through one certification and understand the what and why to be effective in a paid job.

What did these developers take away from the hackathon?

The hackathon provided AWS GPU credits so that the devleopers could run simulations. That was really cool and a few of them got tickets to come to the F8 developers conference.

The data science talk

I detailes my journey into data science and Facebook which some people were inspired by. Learning skills online is really cool and useful.

Survey

I decided I would this group as my target for understanding engagement and trying to determine whether I could increase engagement by providing meaningful contet targeted to this specific group. I created a survey using a google form which I placed in the group to get feedback from these developers which would let us know how exactly we could help them and what specific resources they were looking for.

After reviewing the responses from this group most of the members in the group were interested in deep learning and reinforcement learning and wanted more reources from the Facebook reserach group. I sent messages to Yann LeCunn, who is the AI (artificial intelligence) research lead at Facebook (Meta) and he shared some blog posts and other online resources that I could share.

The marketing team lead realized a few months before I joined the team the need for more resources for developers who are interested in research careers and understanding how large organizations were using and applying their research. Currently in 2022, there is somuch documentation that people can use to understand what is going on with Meta Research, there is an open blog and lots of technical talks. At this point in time in 2018 the research group created 6 lightning talks to the Dev-c community breaking down the high level concepts in deep learning and was trying to figure out more ways to distribute this information to more people in a way that was appropriate for early career reserachers.

Understanding how important content was for this community did shape the product that came out of this research, the content management system, which is written about in another post

The pre-event Guide (CMS)

As a team, developer relations, we realized that providing content to the Dev-C leads was very important for them to have successul in person events because these developers were not experts with any of the APIS’s they were consuming or interested in using.

Our team was full of program managers and marketing specialists so we did not have the technical domain expertise we would need to supply appropriate content or documentation to the developers. The market has changed a lot over time and more technical companies are hiring technical writers who are creating incredible documentation so that it’s much easier for developers to get started.

Why is this a problem that a sharing platform could not solve?

the way that search is set up in groups…

Lets say that a developer in your region in your city posts content about meta-learning in your group on January 3, 2019. Today is April 4, 2022 and in your group there have been so many new posst probably on the same topic so how would you track down that specific post? If you took the time to parse through text search and all of the results that display when you type meta=learning you would still have to go back 3 years to find that data and thats a lot. You would have to read through every post and validate whether the content posted is from a credible source and actually solves the problem you are having. Many people write things on the internet that are not true and are not useful.

This is the problem I tried to solve for this small group by creating a document, a google document, which included links from credible soures in articial intelligence research and I tried to label the documents by topic area so that it was clear what the document was useful for.

The project plan

The purpose of the trip was to collect information about the developers in this region and understand how we as a program could support these developers more effetively. All of the other program managers and team members had many opportunities to travel abroad and understood what it was like to live in the developing world. I lacked all of that context having grown up in a major city in America my entire life. I needed to prove to my team that I was capable of using the information I would gather from the trip to improve online engagement in the group.

All of the other parts of this write-up were section in the project plan including:

After the event I updated my monthly analytic engagement report which sumarized the online engagement in the group and I found that one week after the trip engagement increased 400% online. The engagement was meaningful and most of if was relatd to deep learning content. I presented my findings to senior leadership and the content management system project which was on hold was approved for immediate release and the product was announced at F8 two months later.

On the ground interviews

I recorded many talks with developers which you are free to watch and learned a ton from their personal experiences. When I think about the way that I look at the working world post-COVID-19 and this project, I am reminded of how far technology can take you and how if you create a digital resource, it could potentially help someone else in the future with less access than you.

5 Steps to deigning your own user research study

  1. Find a question out in the world which has yet to be answered in a way suitable for me or someone else
  2. Explore ways in which other people have tried to solve the problem before.
    • If someone has successfully designed something similar in the past, think through how it could be improved
    • If you are unable to find an existing similar solution, explore why others did not find the space worthy enough to build in and weigh the costs and benefits to spending time and resources building in said space
  3. Chat with as many product owners or end users as possible to get feedback and talk about your understanding of the space
    • Get these people to share or create images or websites/ groups
    • Ask them for recommendations
  4. Create a project plan
  5. Talk openly about your project and share the project details document with appropriate parties.
    • The more openly you talk about the project you will find other people who have an interest in the outcome of the project and may be able to allocate financial resources to the project