Prepare to run studies against EMIS

Đang mở
  • Có gì không ổn?
#8 1 bình luận 0 reaction 0 người được giao Xem trên GitHub

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
15/100
Loại issue
Tính năng
Độ rõ ràng
Cần làm rõ
Mức độ hoạt động
Đình trệ
Công nghệ
git, github, python, ubuntu
Lĩnh vực
cli, devops, infrastructure

Hướng nghiên cứu

Issue không nêu tên tệp hoặc kiểm thử cụ thể nào và trải rộng trên job-server, job-runner, cơ sở hạ tầng EMIS và cohort-extractor. Hãy bắt đầu bằng việc xem xét các phần đó cùng với playbooks/EMIS.md; cần thống nhất một phạm vi được giới hạn rõ ràng trước khi triển khai, trong đó mỗi khu vực được chọn phải có các yêu cầu về thiết lập, backend hoặc workflow được xác định.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

In order to run studies against EMIS with the same workflow that we use with TPP, we need to make changes to several parts of the system.  This is my initial braindump of what we need to do and I'm sure I've missed things. Please edit to add your notes, and then let's discuss in a call on Tuesday or Wednesday this week.

  • job-server (@ghickman)
    • users need to be able to select which backend they run jobs
      • initially, we want to hide the ability to run against EMIS by default, and have specific opt-ins for supported repos. Longer term we will need to bake-in the concept of which backends are required or able to run a given study definition (see below). Initial implementation could just be a database flag which is false by default and we manually enable for specific repos.
    • /status/ will need to show information about multiple backends
    • Think about error reporting / sentry / etc
    • questions:
      • should a user be able to run a job against multiple backends? (@sebbacon's best placed to know what users will expect)
        • my (Seb's) view is that UI support would be via an extension of the usual "run" mechanism; "run all" would select everything as it currently does, but there are now N * M tickboxes where N is backends and M is actions. Users could uncheck a specific backend if they wanted (actions would be grouped visually by backend in the UI); if they want to completely exclude a backend from the UI they'd do this by editing their project.yaml (see below). "Supported backends" would show somewhere obvious on a workspace header.
  • job-runner (@evansd)
    • possibly no code changes?
    • longer term we will need to bake-in the concept of which backends are required or able to run a given study definition. This will probably be on a per-column basis; you might be able to extract patient ages in emis but not SGSS status, for example. Users should also be able to define which backends are included or excluded in their project.yaml, i.e. to be able to skip TPP backend completely (for example, they just don't need it, so it's faster)
    • we'll need to flesh out playbooks/EMIS.md
  • EMIS infrastructure (@bloodearnest)
    • user management (lower priority)
      • set up unix groups for level 2/3/4 access
      • create user accounts
        • level 2 access should be limited to a handful of engineers with NHSE contracts
        • level 3 access should be limited to researchers with appropriate NHSE contracts
        • level 4 access can be wider
          • what are the requirements here?
      • set up directory structure necessary for high/medium privacy outputs
    • ensure that we can pull repos from GH
    • ensure that we can push output to GH
    • ensure users can install opensafely cli tool and os-release script (see docs.opensafely.org)
    • harden and buildout software installations
      • consider asking for an ubuntu container within which we have root?
      • job runner as a service - systemd?
      • scripted installation - at least the basics we can build on, supplemented by installation narrative if needed as stopgap
      • backups if necessary
      • log rotation, disk space monitoring, root cron emails setup, etc, if we have root access. If not, conversation with EMIS about what they've set up
    • work out how to support viewing, editing, publishing outputs
      • For viewing outputs, a web browser should be fine (pdfs, svgs, html and text)
      • For diffing outputs, command line git may be sufficient but visual would be ideal: Github Desktop or at a push, gitweb?
      • For redacting outputs, a text editor is needed. We could consider mandating VS Code for simplicity, but worth canvassing
      • For publishing outputs, we may need to install Github Desktop, although again we might be able to mandate command line git (if we provide adequate documentatin)
      • I think it boils down to either (a) provisioning a Windows review server with access to L4 data or (b) providing a web browser and expecting command line tool usage. And I think (a) is probably unavoidable
    • questions:
      • can we be responsible for creating user accounts?
  • cohort-extractor (@inglesp)
    • questions:
      • do we need to pass EMIS_ORGANISATION_HASH in as an environment variable, or can it be hard-coded?
      • how do we support studies that use backends with different coding systems?
        • it would be wonderful if we could use SNOMED in TPP
      • how can we support more features from the TPP backend (eg date expressions) in EMIS?
      • is there a half-way house for things we're willing to implement now while we await refactor?
Ngôn ngữ chính
Python
Star
0
Fork
0
Merge trung bình
12 giờ 47 phút
Pull request đã merge (30 ngày)
2

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của opensafely-core/sysadmin

Tất cả issue của opensafely-core/sysadmin

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.