Setting Up the Basic Structure for DeepResearcher | CodeSignal Learn

Setting Up the Basic Structure for DeepResearcher | CodeSignal Learn

Introduction: Laying the Foundation for DeepResearcher

Welcome to the first lesson of the " Creating a Researcher in Python with OpenAI " course! In this course, you will learn how to build DeepResearcher, an AI-powered research tool that can search the web, gather information, and generate a final report — all using Python . Before we dive into the details of web searching and AI, it’s important to set up a solid foundation. A clear project structure will help you keep your code organized, make it easier to add new features, and help you debug problems as you go. In this lesson, we’ll walk through the basic structure of the DeepResearcher project and explain how the main program is set up. By the end of this lesson, you’ll understand how the main parts of the project fit together and be ready to start building out each piece in future lessons.

Recall: Project Flowchart

Let’s start with a quick reminder about how the DeepResearcher works: This flowchart illustrates the step-by-step process, showing how each component fits into the overall workflow. The LLM (Language Model) and Web components work together to automate the research process.

Understanding the Main Program

Now, let’s look at the main file of our project: main.py . This file is the entry point for DeepResearcher . It’s where the program starts running. Let’s break down the key parts of this file step by step.

1. Importing Functions

At the top of the file, we import a function from another part of our project: Python from deepresearcher.web.web_searcher import clear_visited_pages This allows us to use the clear_visited_pages function in our main program.

Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

2. Defining Function Stubs

Next, we see several function definitions. Right now, these functions are just “stubs” — they don’t do anything yet, but they show what the main steps of our program will be. def generate_initial_search_queries(user_query: str): pass def perform_iterative_research(user_query: str, new_search_queries: list, all_search_queries: list, iteration_limit: int): pass def generate_final_report(user_query: str, contexts: list): pass generate_initial_search_queries : This will take the user’s research topic and create a list of search queries. perform_iterative_research : This will handle the main research loop, searching the web and collecting information. generate_final_report : This will take all the information we’ve gathered and create a final report. The pass statement is a placeholder. It means “do nothing for now.” We’ll fill in these functions in later lessons.

3. The Main Function

The main logic of the program is inside the research_main() function: def research_main(): user_query = input("Enter your research query/topic: ").strip() iteration_limit = input("Max number of iterations (default 10): ").strip() iteration_limit = int(iteration_limit) if iteration_limit.isdigit() else 10 clear_visited_pages() new_search_queries = generate_initial_search_queries(user_query) if not new_search_queries: return all_search_queries = new_search_queries.copy() aggregated_contexts = perform_iterative_research(user_query, new_search_queries, all_search_queries, iteration_limit) generate_final_report(user_query, aggregated_contexts) Let’s break this down: The program asks the user for a research topic and how many times to repeat the research process. It clears any previously visited web pages. It generates the first set of search queries. If there are no search queries, it stops. It copies the search queries to keep track of all queries used. It performs the main research loop. Finally, it generates a report.

Python

def generate_initial_search_queries(user_query: str):
    pass

def perform_iterative_research(user_query: str, new_search_queries: list, all_search_queries: list, iteration_limit: int):
    pass

def generate_final_report(user_query: str, contexts: list):
    pass

Python

def research_main():
    user_query = input("Enter your research query/topic: ").strip()
    iteration_limit = input("Max number of iterations (default 10): ").strip()
    iteration_limit = int(iteration_limit) if iteration_limit.isdigit() else 10

    clear_visited_pages()
    new_search_queries = generate_initial_search_queries(user_query)
    if not new_search_queries:
        return

    all_search_queries = new_search_queries.copy()
    aggregated_contexts = perform_iterative_research(user_query, new_search_queries, all_search_queries, iteration_limit)
    generate_final_report(user_query, aggregated_contexts)