Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Exploratory Analysis on MyLA311 Service Request Data 2020

オープン
  • 何か問題がありますか？
#83 コメント 14 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
25/100
issue の種類
機能追加
明瞭さ
説明が足りない
活発さ
停滞
技術スタック
jupyter-notebook
領域
data, data-visualization

調査の方向性

リンクされている Google Colab ノートブックと MyLA311 Service Request Data 2020 データセットから始めてください。関連する所得、人口、または地区のデータを含め、明確に定義した落書き分析の範囲を選択する前に、既存のクリーニングと EDA を確認してください。選択した関係が分析・文書化され、明確な知見によって裏付けられれば完了です。

索引モデルが issue の本文から書いたものです。

説明

feature: workshops guide: 311 data PBV: Educational Module role: data analyst size: 5 pts
Overview

Analyze MyLA311 Service Request Data 2020, a public dataset available for preview and download (1,491,773 records, 34 feature columns), to understand what kind of stories (insight) we could tell based on it

Action Items
  • Dataset preprocessing, overview check NA, duplicated records, and looking for any potential issues with the dataset
  • EDA on interesting features
** possible next step analysis: focus on graffiti (NEW) **

put the data into context & dig in to understand graffiti from "in-depth" (data literacy) data analysis:
(need to find other datasets)

  • relation with income level (wealth, safety, crime)
  • relation with population (maybe only useful to compare similar areas (define "area") )
  • aggregate by council districts
  • possibly tag some areas (university, tourism, downtown, residency, commercial) -> could be difficult to find the dictionary
Analysis & Findings
  1. 104 NC, 203 NC names, more than the actual 99 NC's
  2. Find that ServiceDate and ClosedDate have NA values
  3. Find one request that has a typo of ServiceDate and ClosedDate (year=3020)

------- (Analysis below is based on requests (94.96%) with valid CreatedDate, UpdatedDate, ServiceDate and ClosedDate )-------

  1. Compare distribution of CreatedDate, UpdatedDate, ServiceDate and ClosedDate by hours
    • 1,278,193 (90.23%) requests have ServiceDate during 00:00 to 00:59
  2. Compute new columns: duration between CreatedDate and each of UpdatedDate, ServiceDate and ClosedDate
  3. Find that there are requests with process hours <0
  4. Compare
    • ratios of requests with ServiceData on the same day as CreatedDay in total requests of a type
    • ratios of requests of a type in total requests
  5. Analysis on Graffiti Removal, the request type that has the most ServiceDate later than CreatedDate
    • by processing hours
    • by processing days
  6. Count of RequestType, RequestSource and Status
Stories

  • Make Suggestions to the database (based on merely analyzing if the values of certain features make sense):

    1. NC names need more consistency: e.g ''GRANADA HILLS NORTH NC" & "Granada Hills North" ; "SYLMAR NC" & "Sylmar"
    2. Convert timestamp to 24-hour time to facilitate data analysis
    3. New mechanism for ServiceDate & ClosedDate (possibly, some systems record the ServiceDate to 00:00 on the date of service by default, creating problems when considering processing days or hours if w/o explanations)
    • RequestSource: Driver Self Report, Voicemail
    • RequestType: Graffiti Removal, Multiple Streetlight Issue, Single Streetlight Issue
    • Status: all
    • PolicePrecinct: CENTRAL, NEWTON
    • AssignTo:
    • Owner: BSL, OCB

  • Graffiti Removal Processing Days:

    • Understand how to look at boxplot and find outliers by IQR
    • Aggregate hours to days to have more insights
    • 91.40% Graffiti Removal requests are processed within 2 days after the day request created
    • 67.68% Graffiti Removal requests are processed on the same day
    • 3.2% Graffiti Removal requests are processed after a week
    • 0.79% Graffiti Removal requests are processed after a month
    • 0.09% Graffiti Removal requests are processed after a year

------------------------------------UPDATE------------------------------------

Data cleaning

each record has 4 timestamp columns: CreatedDate, UpdatedDate, ServiceDate, ClosedDate

  • valid data: all the timestamp columns are in 2020 and ServiceDate>CreatedDate and ClosedDate>CreatedDate
  • NA data: records with NA (missing value)
  • NC: some NC's are not in the LA NC's area but still in the data and 1 NC is misclassified: remove NC's not in the LA NC's area and correct the misclassified one
Analytical concepts:
  • how outliers affect data and the difficulty of dealing with it; average, median
  • how to define, find and remove outliers
  • by comparing simple bar plots, we could derive many insights
  • differences between a bar plot and histogram
  • how to deal with missing values and invalid data: don't make assumpting of missing data before knowing why they are missing. missing values in categorical features could be treated as another level, which can be used to derive insights
  • understanding meaning of a column before analysis
Google Colab Notebook

data cleaning
EDA

主要言語
言語のデータがありません
スター
18
フォーク
3
30日以内にマージされた PR はありません

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

hackforla/access-the-data のほかの issue

hackforla/access-the-data の issue をすべて見る

似ている issue

Data Engineering の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。