Business Analytics Course [2026] | Master Business Strategy & Data-Driven Decisions

📖 Lesson 3.2 — Data Collection Methods (Surveys, Databases, APIs)

Collecting the right data is the first practical step in analytics. This lesson covers common collection methods — surveys, databases, APIs — plus logs, sensors, web scraping and third-party sources, with pros, cons, and implementation tips.

Data collection should always begin with the problem statement: what do you need to measure to answer the question? Once you know what you need, choose methods that are reliable, cost-effective, and ethical. Below we unpack the most commonly used methods and when to use each.

📝 Surveys — Collecting Primary Feedback

Surveys are ideal for capturing opinions, satisfaction, preferences, and self-reported behaviour. They are a primary data source — you design the questionnaire to collect exactly what you need.

When to use

Gather customer satisfaction (NPS) or feedback
Capture preferences not visible from behavioural data
Validate hypotheses from analytics

Pros & Cons

Pros: Customizable, direct, focused
Cons: Response bias, low response rates, cost/time to run

Practical Tips

Keep surveys short (5–10 questions) and mobile-friendly.
Mix closed (quant) and open (qual) questions when needed.
Use random sampling and incentives carefully to improve response rates.

🗄️ Databases — Internal Transactional & Master Data

Databases are the backbone of most analytics workflows. Internal relational databases, data warehouses, and data lakes store sales, customer, HR, and financial records.

Common Types

OLTP databases (transactions)
Data warehouses (aggregated historical data)
Data lakes (raw files, logs)

Pros & Cons

Pros: High reliability, structured, repeatable
Cons: Integration complexity, stale if not updated, requires governance

Practical Tips

Define stable keys (customer_id, order_id) for joins.
Use a central data dictionary for field definitions.
Schedule regular ETL/ELT jobs to keep warehouse fresh.

🔗 APIs — Connecting to External & Third-Party Data

APIs (Application Programming Interfaces) allow systems to request data from external services — such as payment gateways, social platforms, weather services, or logistics partners — in near real-time.

When to Use

Enrich internal data with external signals (e.g., weather, ratings)
Fetch transaction data from payment providers
Automate periodic retrievals for dashboards

Pros & Cons

Pros: Real-time or frequent updates, rich external signals
Cons: Rate limits, API changes, auth/security overhead

Practical Tips

Use caching to reduce API calls and handle rate limits.
Monitor API failures and fallback gracefully.
Document versioning and update processes for API changes.

📡 Logs, Sensors & Web Scraping

Many modern sources produce machine-generated data: server logs, IoT sensors, clickstream events. Web scraping extracts public web data when APIs are unavailable (ensure legal compliance).

Server & application logs: event timing, errors, performance metrics
IoT & sensor data: telemetry for equipment or environment monitoring
Web scraping: product prices, reviews, public job listings (use responsibly)

Practical Tips

Standardize timestamps/timezones when combining log data.
Compress and partition high-volume logs for cost-efficient storage.
Respect robots.txt and legal constraints for web scraping.

⚖️ Sampling, Coverage & Bias

Data collection design must consider representativeness. Sampling errors and selection bias can mislead analysis. Always ask: Does this data represent the population of interest?

Random vs. stratified sampling
Non-response bias in surveys
Coverage gaps in internal systems (legacy data missing)
Survivorship bias and its effect on conclusions

Mitigation Tips

Use weighting or stratification when survey response is uneven.
Cross-check external sources to fill internal coverage gaps.
Document known biases and account for them in conclusions.

🔒 Ethics, Privacy & Legal Considerations

Always respect data privacy and legal constraints: obtain consent for personal data, anonymize where necessary, and follow regulations (GDPR, local privacy laws). Secure sensitive data in transit and at rest.

Collect minimum necessary data (data minimization)
Use anonymization or pseudonymization for personal identifiers
Maintain an audit trail of data access and processing

📌 Data Collection Checklist

Define exact fields and formats needed (schema)
Choose sources that match the problem and timeline
Plan sampling strategy and expected coverage
Consider privacy, consent & legal compliance
Document ETL and collection frequency (real-time vs batch)

Action Task

List 3 potential data sources for a retention study (internal transaction logs, customer survey, social reviews). For each, note pros/cons and one mitigation for bias.

➜