Video summary

РЕАЛЬНОЕ СОБЕСЕДОВАНИЕ Data Scientist Иннотех - Middle

Main summary

Key takeaways

News and Commentary

Overview

The video is a “real interview” for a Middle Data Scientist role. The interviewee discusses prior ML projects and practical engineering considerations such as deployment, performance, and tooling, followed by a short Q&A about the company’s work setup and growth policies.

Key points from the interviewee’s experience

Role focus

  • The candidate positions themselves as a Middle DS specialist.
  • Background emphasized: classic ML.
  • Transformer-based models are used mainly as feature generators, not as end-to-end LLM-style systems.

Pharma product name matching project (“Aurora”)

Problem

  • The company DCMGUP receives drug names from pharmacies.
  • These names must be matched to an internal reference database.
  • Goal: reduce dependence on human copywriters who currently perform manual mapping/markup.

Goal

  • Automate matching.
  • Reduce manual labeling and related copywriter markup cost.

Approach: 2-stage pipeline

  1. Feature generation

    • Use a RoBERTa model to embed/represent drug names.

  2. Candidate reduction and ranking

    • Use FaceIn (search/indexing) to retrieve the 500 most similar candidates.
    • Select top 10 using cosine similarity.
    • Apply a second BERT-based probability ranking to choose the best match.
    • Decision rule:
      • If top probability exceeds a threshold (~95%), accept automatically.
      • Otherwise, route to a copywriter.

Impact

  • Improved copywriter efficiency by approximately 10–15%.
  • Measured by comparing average batch processing time, assuming part of the workload was handled by ML.

Scalability and performance discussion

  • The candidate explains that cosine-only solutions became too slow at large scale (tens of millions of rows).
  • Switching to FaceN-style retrieval improved runtime.

Hypothetical “billion-event scale” inference ideas

  • Fine-tune Russian text transformer variants (e.g., S-BERT / RuBERT approaches).
  • Improve throughput using:
    • Parallelization via containers/microservices and request batching (conceptually multiple containers behind a load balancer).
    • They admit limited direct parallelism experience, but suggest multiprocessing or containerized distribution as possible options.

Model deployment & MLOps

  • Primary deployment: an online service (accept requests, return results).
  • Packaging: Docker, then handed off to another team for rollout.
  • Mentioned local model versioning/guarding, though details were partially unclear.
  • Retraining project:
    • Automated weekly retraining using Airflow when new data arrives.

Additional project example (bank + agricultural equipment)

  • Built a pipeline to:
    • Scrape equipment characteristics and prices from websites.
    • Convert parsed content into tabular form.
    • Use ML/EML to estimate price.
  • New parsed data triggered further training in a retraining loop.

Tech/tooling Q&A

Frameworks

  • Limited experience with TensorFlow (not much work there).
  • Differentiation point mentioned: dynamic vs static computation graphs (with PyTorch referenced).

Spark

  • Used Spark only rarely.
  • Notes suggest Spark can be helpful within the broader ecosystem and for table-like operations.

Training memory limits

  • Discussed handling datasets that don’t fit in memory by splitting/chunking training on parts or subsets.

LLM/NLP direction

  • Expresses interest in gradually expanding toward NLP/LLMs, though not deeply yet.

Company/role logistics and candidate expectations

Work setup (company pitch)

  • Full remote work
  • Laptop provision (Windows / MacBooks)
  • Standard benefits/perks mentioned:
    • cafeteria/discounts
    • sports/medical-type benefits
    • partner discounts

Location and signing documents

  • Company has multiple offices mentioned: Moscow, Sochi, St. Petersburg, Voronezh
  • For non-Moscow candidates, document signing may involve courier/laptop handling (details to be clarified).

Team distribution

  • Team members located across multiple cities.
  • Occasional offline workshops/events.

Growth concern (candidate reaction)

  • After the interview, the candidate highlights a critical concern about the company’s salary growth/promotion mechanics.
  • Implied concern: review cycles and feedback may not reliably convert into meaningful advancement.

The tone suggests skepticism about how promotion and compensation growth are operationalized.

Overall takeaway

The interviewee shows solid applied ML experience, especially in practical text matching using an approach like:

  • embeddings + retrieval + probabilistic acceptance threshold

They also demonstrate awareness of operational concerns:

  • Dockerized online deployment
  • automated retraining workflows

However:

  • Their final tone indicates skepticism about the company’s promotion/compensation growth system
  • They also acknowledge limited depth in some areas (e.g., parallelism and certain tooling)

Presenters / contributors

  • Interviewee: Data Scientist candidate
  • Interviewer / recruiter: Company representative

Original video