Research Data Analysis with SciSpace Agent: Step‑by‑Step Guide
Based on SciSpace's video on YouTube. If you like this content, support the original creators by watching, liking and subscribing to their content.
SciSpace Agent supports an end-to-end research workflow: extract structure from PDFs, assess and clean CSV data, run descriptive/inferential statistics, and generate shareable outputs.
Briefing
SciSpace Agent is positioned as an end-to-end workflow tool that turns messy research inputs—PDFs, CSVs, and raw interview transcripts—into structured outputs, cleaned datasets, statistical findings, visualizations, and even publication-ready reports, all with prompts instead of hand-written code. The core value demonstrated is speed plus structure: upload unstructured material, ask for a specific deliverable (JSON metadata, data-quality diagnostics, regression results, thematic themes), and get back consistent, exportable artifacts that can be shared with collaborators.
The walkthrough begins with a simple test using a PDF research paper about “buy now pay later” transactions. After uploading the document, the agent is prompted to extract a five-sentence plain-language summary, key insights, and a metadata table (including title, authors, year, journal, DOI, keywords). The output is returned as a JSON object, illustrating a repeatable pattern: raw text becomes uniform structured data. The interface also logs actions via clickable “yellow badges,” letting users see what the agent did while reading the PDF.
Next comes the analytic phase, starting with a real-world problem: unclean data. A CSV of data science job postings scraped from “blast dot” is described as explicitly messy. Rather than immediately cleaning, the agent performs a comprehensive data quality assessment—producing a data structure overview (row/column counts, column names, apparent data types), sample previews (first and last rows), and checks for completeness, consistency, accuracy, and formatting. The assessment flags multiple issues: company names are contaminated (with ratings embedded), logical duplicates exist (same job in the same company), salary formats are highly inconsistent (30+ formats), and missing value encodings appear. The agent then generates a Python script, runs it, and provides an executive summary plus dashboards and charts summarizing the problems.
After diagnosing the dataset, the agent is prompted to clean it in priority order based on the earlier findings. It writes and executes the cleaning code, applies transformations such as standardizing salary data and handling missing values, and outputs a final cleaned CSV along with a cleaning dashboard showing what changed.
The workflow then shifts to quantitative analysis using a medical study dataset (30 patients; 20 treatment, 10 control) with baseline and follow-up systolic/diastolic blood pressure, medication adherence, side effects, smoking status, BMI, and quality-of-life scores. The agent produces descriptive statistics and baseline comparisons, then moves into inferential testing to confirm whether blood pressure reduction differs between groups. Results highlighted include statistically significant reductions in both systolic and diastolic pressure for the treatment group, with gender differences in response, while medication adherence and smoking status do not significantly affect efficacy in this dataset. It also flags limitations such as small sample size and missing data.
Finally, the agent extends beyond numbers into qualitative research. Using a CSV of interview transcripts, it performs inductive thematic analysis (six primary themes with 18 sub-themes), provides theme definitions and quote-level examples, and generates visualizations like heat maps and network maps. It then runs sentiment analysis with role-based breakdowns, and builds an interactive, publication-oriented web dashboard (HTML/CSS/vanilla JavaScript with Chart.js) for stakeholder sharing. The session closes with an automated LaTeX-ready, production-style research report compiled into downloadable files, reinforcing the “prompt-to-paper” promise.
Overall, the demonstration frames SciSpace Agent as a practical pipeline for research teams: extract structure from unstructured sources, diagnose and repair data quality, run descriptive and inferential statistics, model relationships (including multiple regression), and package findings into shareable visuals and reports—without requiring users to write code themselves.
Cornell Notes
SciSpace Agent is demonstrated as a prompt-driven pipeline that converts research inputs into usable outputs across the full analysis lifecycle. It first extracts structured information from unstructured PDFs into formats like JSON (summary, key insights, and metadata). It then handles messy CSV data by running automated data-quality diagnostics (completeness, consistency, accuracy, formatting), generating and executing Python scripts, and producing dashboards. After cleaning, it performs descriptive and inferential statistics on a medical dataset, highlighting significant blood pressure reductions for treatment and gender-linked response differences while noting limitations like small sample size. For qualitative work, it performs inductive thematic analysis and sentiment analysis on interview transcripts and can compile results into an interactive dashboard and a LaTeX-ready research report.
How does the agent turn a PDF research paper into structured, reusable data?
What does a “comprehensive data quality assessment” include, and what kinds of problems were found in the job postings CSV?
How does the workflow shift from diagnosing data problems to producing a cleaned dataset?
What statistical workflow was used for the medical study dataset, and what conclusions were highlighted?
How does the agent handle qualitative research inputs like interview transcripts?
What does “sharing” look like after analysis—dashboards, exports, and reports?
Review Questions
- When extracting from a PDF, what specific structured outputs were requested (and in what format) to make the results reusable?
- What categories of data quality issues did the agent check for in the job postings CSV, and which two issues were described as most critical?
- In the medical study analysis, which variables were reported as significantly affecting treatment response, and which were reported as not significantly impacting efficacy?
Key Points
- 1
SciSpace Agent supports an end-to-end research workflow: extract structure from PDFs, assess and clean CSV data, run descriptive/inferential statistics, and generate shareable outputs.
- 2
Prompting for structured formats (like JSON) enables consistent metadata and summaries from otherwise unstructured text.
- 3
For messy datasets, the agent can run a comprehensive data-quality assessment first—then generate and execute Python code to clean data in priority order.
- 4
In the medical study example, the agent highlighted significant blood pressure reductions for treatment and gender-linked response differences while flagging small sample size and missing data as limitations.
- 5
For qualitative datasets, the agent performs inductive thematic analysis with hierarchical themes, quote-level examples, and role-based theme mapping.
- 6
The agent can produce interactive stakeholder dashboards (HTML/vanilla JavaScript with Chart.js) and compile LaTeX-ready research reports for publication workflows.
- 7
Action logs in the UI (e.g., clickable badges) help users audit what the agent did during extraction and analysis.