Prepare to run studies against EMIS

Abierto
  • ¿Algo no está bien?
#8 1 comentario 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
15/100
Tipo de issue
Nueva funcionalidad
Claridad
Necesita aclaración
Estado de actividad
Estancado
Stack tecnológico
git, github, python, ubuntu
Área
cli, devops, infrastructure

Línea de trabajo

El issue no nombra archivos ni pruebas específicos y abarca job-server, job-runner, la infraestructura de EMIS y cohort-extractor. Empieza revisando esas secciones y playbooks/EMIS.md; se necesita un alcance acordado y delimitado antes de la implementación, con los requisitos de configuración, backend o flujo de trabajo definidos para cada área seleccionada.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

In order to run studies against EMIS with the same workflow that we use with TPP, we need to make changes to several parts of the system.  This is my initial braindump of what we need to do and I'm sure I've missed things. Please edit to add your notes, and then let's discuss in a call on Tuesday or Wednesday this week.

  • job-server (@ghickman)
    • users need to be able to select which backend they run jobs
      • initially, we want to hide the ability to run against EMIS by default, and have specific opt-ins for supported repos. Longer term we will need to bake-in the concept of which backends are required or able to run a given study definition (see below). Initial implementation could just be a database flag which is false by default and we manually enable for specific repos.
    • /status/ will need to show information about multiple backends
    • Think about error reporting / sentry / etc
    • questions:
      • should a user be able to run a job against multiple backends? (@sebbacon's best placed to know what users will expect)
        • my (Seb's) view is that UI support would be via an extension of the usual "run" mechanism; "run all" would select everything as it currently does, but there are now N * M tickboxes where N is backends and M is actions. Users could uncheck a specific backend if they wanted (actions would be grouped visually by backend in the UI); if they want to completely exclude a backend from the UI they'd do this by editing their project.yaml (see below). "Supported backends" would show somewhere obvious on a workspace header.
  • job-runner (@evansd)
    • possibly no code changes?
    • longer term we will need to bake-in the concept of which backends are required or able to run a given study definition. This will probably be on a per-column basis; you might be able to extract patient ages in emis but not SGSS status, for example. Users should also be able to define which backends are included or excluded in their project.yaml, i.e. to be able to skip TPP backend completely (for example, they just don't need it, so it's faster)
    • we'll need to flesh out playbooks/EMIS.md
  • EMIS infrastructure (@bloodearnest)
    • user management (lower priority)
      • set up unix groups for level 2/3/4 access
      • create user accounts
        • level 2 access should be limited to a handful of engineers with NHSE contracts
        • level 3 access should be limited to researchers with appropriate NHSE contracts
        • level 4 access can be wider
          • what are the requirements here?
      • set up directory structure necessary for high/medium privacy outputs
    • ensure that we can pull repos from GH
    • ensure that we can push output to GH
    • ensure users can install opensafely cli tool and os-release script (see docs.opensafely.org)
    • harden and buildout software installations
      • consider asking for an ubuntu container within which we have root?
      • job runner as a service - systemd?
      • scripted installation - at least the basics we can build on, supplemented by installation narrative if needed as stopgap
      • backups if necessary
      • log rotation, disk space monitoring, root cron emails setup, etc, if we have root access. If not, conversation with EMIS about what they've set up
    • work out how to support viewing, editing, publishing outputs
      • For viewing outputs, a web browser should be fine (pdfs, svgs, html and text)
      • For diffing outputs, command line git may be sufficient but visual would be ideal: Github Desktop or at a push, gitweb?
      • For redacting outputs, a text editor is needed. We could consider mandating VS Code for simplicity, but worth canvassing
      • For publishing outputs, we may need to install Github Desktop, although again we might be able to mandate command line git (if we provide adequate documentatin)
      • I think it boils down to either (a) provisioning a Windows review server with access to L4 data or (b) providing a web browser and expecting command line tool usage. And I think (a) is probably unavoidable
    • questions:
      • can we be responsible for creating user accounts?
  • cohort-extractor (@inglesp)
    • questions:
      • do we need to pass EMIS_ORGANISATION_HASH in as an environment variable, or can it be hard-coded?
      • how do we support studies that use backends with different coding systems?
        • it would be wonderful if we could use SNOMED in TPP
      • how can we support more features from the TPP backend (eg date expressions) in EMIS?
      • is there a half-way house for things we're willing to implement now while we await refactor?
Lenguaje dominante
Python
Estrellas
0
Forks
0
Merge medio
12 h 47 min
PR fusionados (30 d)
2

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de opensafely-core/sysadmin

Todos los issues de opensafely-core/sysadmin

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.