Project Overview

Project Purpose and Objectives

The core purpose of this project is to provide a programmatic interface to interact with the Qualtrics survey collaboration system, enabling automated management of survey access and collaboration workflows. The primary goal is to streamline tasks such as logging in to Qualtrics with institutional credentials, adding collaborators to surveys, and submitting collaboration codes without requiring manual web interactions.

Major Functionalities

• Automated Login via Selenium: The system uses Selenium WebDriver to automate logging into Qualtrics using NYU credentials. This includes handling two-factor authentication and browser trust prompts, simulating human-like typing and pause behaviors to avoid detection or failure.

• Collaboration Management: Programmatic methods to add collaborators to surveys with configurable permission levels, and to accept collaboration invitations through collaboration codes.

• Session Management: Maintains authenticated HTTP sessions with appropriate headers and cookies, including the generation of unique request IDs and authentication tokens required by the Qualtrics API.

These functionalities are encapsulated primarily within the `CollaborationClient` class, designed to abstract the complexity of web automation and API interaction into a clean Python interface.

Example Workflows and Use Cases

Use Case: Adding a Collaborator to a Survey

1. Instantiate the CollaborationClient with the appropriate Qualtrics subdomain.

2. Call the login() method to perform an automated login sequence.

3. Use the add_collaborator(survey_id, collaborator_email, permission_level) method to add a new collaborator by email to the specified survey.

4. The client manages session headers and sends a POST request to the Qualtrics API with collaborator details.

This workflow enables an automated pipeline to manage survey access control without manual intervention.

Use Case: Accepting a Collaboration Code

1. After logging in, call enter_collaboration_code(code) with a valid collaboration code.

2. The client submits the code to Qualtrics programmatically, accepting shared survey access on behalf of the user.

This is useful for bulk or automated acceptance of shared surveys.

Stack and Technologies

• Python 3.x: Primary programming language for its extensive ecosystem and ease of web automation.

• Selenium WebDriver: Automates browser interactions to handle login flows including multi-factor authentication and UI prompts.

• Requests Library: Manages HTTP requests for interacting with Qualtrics APIs post-login.

• SciPy (truncated normal distribution): Models realistic human-like delays between actions to mimic natural behavior.

• UUID Module: Generates unique request IDs required by Qualtrics API headers.

• Getpass: Securely prompts for user credentials without echoing input.

• Chrome WebDriver: Specific browser driver for Selenium automation.

These technologies were selected to balance robustness, ease of automation, and compliance with Qualtrics’ web interface behaviors. Selenium handles complex login UI flows that pure HTTP requests cannot, while `requests` library efficiently manages authenticated API calls once logged in.

High-Level Architecture

The project is structured as a Python package (`qualtrics_collaboration`) containing the core client implementation. The architecture revolves around the `CollaborationClient` class, which orchestrates automated login and API interactions.

Component Interactions

• Frontend (Browser Automation): Selenium WebDriver controls a Chrome browser instance to perform login steps on the Qualtrics web UI.

• Backend (API Client): Once logged in, the client uses requests.Session to send API requests with proper authentication headers and cookies.

• Session & State Management: Cookies and headers are synchronized between Selenium and the requests session to maintain authentication state.

• Human Behavior Simulation: Timing and typing delays are introduced to mimic human interaction, improving reliability against bot detection.

graph TB
    Selenium[Selenium WebDriver] -->|Automates Login| QualtricsUI[Qualtrics Web Interface]
    Selenium -->|Extracts Cookies & Tokens| SessionState[Session & Auth State]
    SessionState -->|Provides Auth| APIClient[Requests API Client]
    APIClient -->|POST Requests| QualtricsAPI[Qualtrics Collaboration API]

Developer Navigation

Frontend Developers (Automation Focus)

• Start with qualtrics_collaboration/collaboration_client.py

• Study the _selenium_login method for browser automation flow.

• Review human interaction simulation methods _human_type and human_sleep.

• Familiarize with Selenium WebDriver setup and element interaction patterns.

Backend Developers (API Interaction & Session Management)

• Focus on add_collaborator and enter_collaboration_code methods inside collaboration_client.py.

• Understand how HTTP headers and cookies are generated and managed via _generate_qualtrics_headers and cookies property.

• Review session handling using the requests library.

Contributors Extending Functionality

• Use qualtrics_collaboration/__init__.py to expose the CollaborationClient class.

• Follow existing class structure and method patterns for consistency.

• Add integration tests for workflows like login and collaborator management.

Visual Diagrams

Component Diagram (Architecture Overview)

graph TB
    Selenium[Selenium WebDriver]
    QualtricsUI[Qualtrics Web Interface]
    SessionState[Session & Auth State]
    APIClient[Requests API Client]
    QualtricsAPI[Qualtrics Collaboration API]

    Selenium -->|Automates Login| QualtricsUI
    Selenium -->|Extracts Cookies & Tokens| SessionState
    SessionState -->|Provides Auth| APIClient
    APIClient -->|POST Requests| QualtricsAPI

Workflow Flowchart (Add Collaborator Example)

flowchart TD
    Start[Start: Instantiate Client] --> Login[Call login]
    Login -->|Success| AddCollab[Call add collaborator]
    AddCollab --> SendRequest[Send POST to API]
    SendRequest --> Response{"Response OK?"}
    Response -->|Yes| Success[Collaborator Added]
    Response -->|No| Error[Handle Error]
    Success --> End[End]
    Error --> End[End]

This overview equips developers with a concise yet comprehensive roadmap to understand the project’s purpose, technologies, architecture, and how to effectively contribute or extend its functionality.