Brainstorming ideas doesn’t necessarily require being factual (it can be enhanced by empirical data of course), making it a high value task for LLMs. Here are a couple of useful excerpts from an excellent 2023 paper by Korinek offering numerous practical examples on using generative AI to enhance productivity for ‘micro’ research tasks:
“Girotra et al. (2023) find that GPT-4 outperforms MBA students at a top US business school on all three dimensions in a contest to develop innovative new product ideas. As a result, they argue that the bottleneck in ideation is increasingly shifting from generating to evaluating ideas”
--
“the remarkable speed and volume of responses generated by LLMs during activities like brainstorming, which generates its own distinct form of usefulness. Even if only a single suggestion out of 20 in examples like this proves beneficial, it may make our research significantly more productive”
Generative AI for Economic Research: Use Cases and Implications for Economists
The latter benefit is echoed by Bail (2023: 6):
“Several of the questions it generated were unimpressive or nonsensical. But of the eight questions it proposed, I consider two of them to be fairly good ideas that test the boundaries of the field”
Can Generative AI Improve Social Science?
With improved prompting, continued dialogue and asking GPT to take on multiple personas to help capture more diverse perspectives, the quality of ideation can be improved still further. For example, you could simply ask ChatGPT to suggest ideas for using generative AI to support research, and you may get a useful introductory response. But here’s an example output asking it to take on 4 different academic personas, each ‘obsessed’ (forcing extreme critical personalities tends to help generate more useful insights given the tendency for LLMs to default to sycophantic or bland outputs) with distinct phenomena (efficiency, writing quality, ethics and higher order human cognition), and presented in the form of a fictional focus group, as an alternative medium through which to generate ideas:
Prompt:
You are moderating a focus group discussion among four academic personas, each with a distinct perspective on using generative AI in social science research. Please simulate their conversation, highlighting their unique viewpoints and the interplay of their ideas. The participants are:
1. Dr Efficiency: A data scientist obsessed with optimising research processes and maximizing productivity.
2. Prof. Eloquence: A linguistics expert fixated on improving the quality and clarity of academic writing.
3. Dr Ethics: A philosopher deeply concerned with the ethical implications of AI in research and potential biases.
4. Prof. Cognition: A cognitive psychologist focused on how generative AI can enhance higher-order thinking and creativity in research.
The discussion topic is: 'Innovative ways to leverage generative AI for enhancing social science research methods and outputs.'
Begin the conversation with Dr Efficiency proposing a novel use of generative AI in research, and have the other participants respond, building on or challenging the idea from their unique perspectives. Ensure that each persona's contribution reflects their core obsession while engaging meaningfully with the others' points.
Continue the discussion for at least two rounds, allowing each persona to introduce a new idea and respond to others
---
Output (from Claude Sonnet 3.5)
Dr Efficiency: "Colleagues, I propose we leverage generative AI to revolutionise our data collection and analysis processes. Imagine using generative AI to generate thousands of hypothetical research scenarios, complete with simulated data sets. This would allow us to test and refine our methodologies at an unprecedented scale before conducting actual studies. We could optimise our research designs, identify potential pitfalls, and significantly reduce the time and resources spent on preliminary studies. What are your thoughts on this approach?"
Prof. Eloquence: "An intriguing proposition, Dr Efficiency. While I appreciate the potential for streamlining our research processes, I'm particularly excited about how generative AI could enhance the clarity and persuasiveness of our academic writing. Perhaps we could use this technology to generate multiple versions of our research papers, each tailored to different audiences or publication venues. This could help us communicate our findings more effectively across various academic and public spheres. However, we must ensure that the AI-generated content maintains the nuanced language and complex ideas inherent in our work."
Dr Ethics: "I have serious reservations about both proposals. Dr Efficiency, while your idea could indeed accelerate research, we must consider the ethical implications of relying on AI-generated data. How can we ensure these simulated scenarios accurately represent real-world complexities and diverse human experiences? There's a risk of perpetuating existing biases or creating new ones. And Prof. Eloquence, your suggestion raises questions about authorship and intellectual integrity. If AI is generating multiple versions of our work, at what point does it cease to be truly ours? We need to carefully consider the ethical boundaries of AI involvement in our research and writing processes."
Prof. Cognition: "These are all valid points. From a cognitive perspective, I see potential in using generative AI as a tool to enhance our creative thinking and hypothesis generation. What if we used AI to propose unexpected connections between disparate fields of study? This could lead to novel research questions and interdisciplinary approaches we might not have considered otherwise. However, I agree with Dr Ethics that we must be cautious. The key is to use AI as a supplement to, not a replacement for, human cognition and creativity in the research process."
Dr Efficiency: "Thank you all for your insights. Building on Prof. Cognition's idea, what if we utilised generative AI to enhance our qualitative data analysis processes? We could develop an AI assistant that helps researchers with coding and thematic analysis of interview transcripts or open-ended survey responses. This could significantly speed up the initial stages of analysis and potentially identify patterns that human researchers might overlook."
Prof. Eloquence: "That's an intriguing suggestion, Dr Efficiency. To address potential concerns about the depth of analysis, we could design this AI assistant to generate multiple interpretations of the data, each with different theoretical lenses. This could serve as a starting point for researchers, sparking new ideas while clearly distinguishing between AI-generated insights and original human analysis. This approach could significantly enhance the richness of our qualitative analyses without compromising the researcher's critical role."
Dr Ethics: "I appreciate the effort to leverage AI in a more focused manner, but we must still proceed with caution. Any AI system we implement in our research processes must be transparently documented and open to peer review. We need to establish clear guidelines for disclosing AI involvement in our work, particularly in qualitative analysis where interpretation is key. Additionally, we should consider how these technologies might impact the nuanced understanding of cultural contexts and lived experiences that is crucial in social science research."
Prof. Cognition: "Dr Ethics raises critically important points about transparency and contextual understanding. Perhaps we can approach this challenge by developing AI tools that not only assist in analysis but also prompt researchers to reflect on their own biases and assumptions throughout the coding process. This could include hard coding dialogue trees where the AI tool challenges the researcher's initial interpretations or highlights potential alternative explanations. This approach could not only enhance our analytical capabilities but also foster a more reflexive and critically-minded research community."
Here’s a different example asking it to come up with 20 ideas and considerations for planning a research project involving gang communities in London, emphasising that this is a new research area. This is a great starting point that could save time as well as ‘analysis paralysis’ procrastination in the initial stages:
Prompt:
You are an experienced social researcher with expertise in urban ethnography, criminology, and community studies, particularly focused on gang dynamics in large cities. I'm a novice postgraduate student researcher interested in studying gang communities in East London. Please give me a list of all considerations I should have in mind when planning this research project.
Response:
Here's a comprehensive list of considerations for planning your research project on gang communities in East London:
1. Ethical considerations:
o Informed consent procedures
o Protecting participants' anonymity and confidentiality
o Potential risks to participants and strategies to mitigate them
o Ethical approval from your institution
2. Safety protocols:
o Personal safety measures for fieldwork
o Emergency procedures and contact protocols
o Risk assessment for different research activities
3. Methodological approaches:
o Ethnographic methods (participant observation, field notes)
o In-depth interviews (structured, semi-structured, or unstructured)
o Focus groups
o Surveys (if appropriate)
o Mixed-methods approach
4. Potential challenges:
o Gaining access to the community
o Building trust with participants
o Navigating potential conflicts between different groups
o Managing emotional stress from sensitive topics
5. Key stakeholders to engage:
o Community leaders
o Local law enforcement
o Social workers and youth outreach programs
o Schools and educational institutions
o Local government representatives
6. Background knowledge to acquire:
o History of gang activity in East London
o Socioeconomic context of the area
o Cultural and ethnic dynamics of the community
o Current policies and interventions related to gang activity
7. Strategies for gaining trust and access:
o Identifying and working with gatekeepers
o Volunteering or participating in community events
o Demonstrating genuine interest and respect for the community
o Being transparent about research goals and methods
8. Data collection methods and tools:
o Audio recording equipment (if consent is given)
o Field notebooks
o Secure data storage solutions
o Appropriate software for data analysis (e.g., NVivo, ATLAS.ti)
9. Analytical frameworks to consider:
o Social disorganisation theory
o Strain theory
o Social learning theory
o Intersectionality
10. Potential biases to be aware of:
o Confirmation bias
o Cultural bias
o Selection bias
o Researcher positionality and its impact
11. Ensuring cultural sensitivity:
o Learning about local customs and norms
o Using appropriate language and terminology
o Respecting cultural boundaries and sensitivities
o Involving community members in the research process
12. Legal considerations:
o Understanding laws regarding criminal activities and reporting obligations
o Data protection and privacy laws
13. Funding and resources:
o Identifying potential funding sources
o Budgeting for equipment, travel, and participant compensation
14. Timeline planning:
o Allowing sufficient time for relationship-building
o Factoring in potential setbacks and delays
15. Longitudinal aspects:
o Assessing the potential for a long-term or follow-up study
16. Ethical exit strategy:
o Planning how to conclude the research without negatively impacting the community
17. Reflexivity:
o Maintaining a reflective journal to document your thoughts, feelings, and potential biases throughout the research process
This list covers a wide range of important considerations for your research project. Remember to consult with your supervisors and experienced researchers in the field for more specific guidance tailored to your particular study.
As always, generative AI outputs should be considered assistive and as a starting point for your own further engagement, particularly for areas in which you don't already have strong domain knowledge. Experienced researchers with deep expertise in a given topic can also benefit from using LLMs as a tool to check they haven't overlooked any important considerations before finalising a proposal, presentation or paper. There can often be blind spots to common ethical or methodological issues (or language use, if the intended audience isn't academic for instance) that are taken for granted, which could benefit from being referenced explicitly. Having an LLM do a final check for potential outstanding issues can be a helpful way to catch any such blind spots, for example:
Prompt:
Attached is a research proposal on <topic> which I think is almost ready to go. I’ve been working on it so long that I want a second pair of eyes to sense check and identify any potential lapses. Can you please thoroughly review and identify anything I may have missed which could be useful to include before submitting? The intended audience is <peers in the field / non-expert academic colleagues / government policy makers / lay public etc.> so anything you identify should have that audience in mind.