1. xml 파일로부터 HKCategoryTypeIdentifierSleepAnalysis 태그를 가진 요소를 모두 취합
2. 요소로부터 시작 시간, 종료 시간, 지역 정보를 가져와 data-sleep.csv로 저장
3. 2022년 7월 11일부터 2024년 11월 26일까지, 30분 단위의 블록 리스트를 생성.
4. data-sleep.csv의 각 열을 읽어, 시작 시간부터 종료 시간까지 해당하는 블록에 값 저장.→ 30분 단위의 블록 각각에 대해 해당 수면 시간이 차지하는 만큼의 시간(분) 값을 더함.→ (예) 11시 20분부터 12시 40분까지인 경우, 각 블록은 10, 30, 30, 10으로 채워짐.
5. 블록 리스트를 result-sleep.txt로 저장

   # result-sleep.txt
   0
   0
   30
   30
   ...
   30

6. 블록 리스트를 일별로 48개 단위로 나누어, result-sleep.csv로 저장

   # result-sleep.csv
   2022-07-11,0,0,0,0,0,0,30,30,30,30,30,30,30,30,30,30,30,30,30,30,0,0,0,...
   2022-07-12,0,0,0,0,0,0,0,11,30,30,30,30,30,30,30,30,30,30,30,30,28,0,0,0,...
   2022-07-13,0,0,0,0,0,0,0,0,0,10,30,30,30,30,30,30,30,30,30,30,0,0,0,0,0,0,...
   2022-07-14,0,0,0,0,0,0,0,24,30,30,30,30,30,30,30,30,30,30,30,30,9,0,0,0,0,...

1. xml 파일로부터 HKQuantityTypeIdentifierStepCount 태그를 가진 요소를 모두 취합
2. 요소로부터 시작 시간, 종료 시간, 걸음 수를 가져옴
3. 시작 시간과 data-region.csv를 이용하여 지역 정보를 가져옴
4. 가져온 정보를 data-walk.csv에 저장
5. data-walk.csv의 각 열을 읽어, 시작 시간부터 종료 시간까지 해당하는 블록에 값 저장.
   → 30분 단위의 블록 각각에 대해, 블록 당 평균 걸음수만큼을 합산.
   → (예) 11시 20분부터 12시 40분까지 40걸음을 걸은 경우, 블록은 10, 10, 10, 10으로 채워짐.
6. 블록 리스트를 result-walk.txt로 저장

   # result-walk.txt
   559
   823
   ...
   8
   605
   1745

7. 블록 리스트를 일별로 48개 단위로 나누어, result-walk.csv로 저장

   # result-walk.csv
   2022-07-11,0,180,479,290,0,8,0,559,823,1494,344,694,303,8,605,1745,763,...
   2022-07-12,1943,0,1366,0,0,947,457,0,0,0,0,72,438,153,0,107,527,996,1628,...
   2022-07-13,0,1563,0,502,905,0,1346,382,0,0,1184,0,0,0,93,92,0,0,95,1336,...
   2022-07-14,0,0,597,92,0,0,46,0,0,1100,701,447,2548,1077,262,0,120,0,0,0,0,...

1. result-sleep.txt와 result-walk.txt로부터 수면 데이터 리스트를 생성

   sleep_list = [0, 0, 0, 0, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, ...]
   walk_list = [0, 0, 0, 0, 0, 180, 479, 290, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...]

2. 오후 10시를 기준으로, 다음날 오후 10시까지 데이터를 모두 수집. 이후 수면 시간이 나타날 때까지 데이터를 추가.
   오후 10시가 기준인 이유는 모든 수면은 오후 10시 이후에 이루어졌기 때문.
   또한 데이터를 추가함으로써 오후 10시 ~ (다음 날 수면 직전)의 데이터를 모을 수 있음.(x, y)는 해당 블록동안 x분 수면, y 걸음을 걸었음을 의미.

   sleepwalk_list[0] = [(0, 0), (0, 180), (0, 479), (0, 290), (0, 0), ...]
   sleepwalk_list[1] = [(0, 410), (0, 410), (0, 0), (0, 0), (0, 1943), ...]
   sleepwalk_list[2] = [(0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 1563), ...]
   sleepwalk_list[3] = [(0, 1410), (0, 771), (0, 0), (0, 0), (0, 0), ...]
   sleepwalk_list[4] = [(0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), ...]

3. 이렇게 만들어진 데이터를 result-sleepwalk.json의 형태로 저장.

   // result-sleepwalk.json
   {
   "2022-07-11": [[0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 180], ...],
   "2022-07-12": [[0, 410], [0, 410], [0, 0], [0, 0], [0, 1943], ...], 
   "2022-07-13": [[0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 1563], ...], 
   "2022-07-14": [[0, 1410], [0, 771], [0, 0], [0, 0], [0, 0], [0, 0], ...],
   ...
   }

# %% [markdown]
# Let's collect sleep data from xml data!

# %%
import xml.etree.ElementTree as ET
import csv

tree = ET.parse("../export.xml")
root = tree.getroot()

with open("data-sleep.csv", "w", encoding="utf-8-sig", newline="") as fp:
writer = csv.writer(fp)

for child in root:
  if (
      child.tag == "Record"
      and child.attrib["type"] == "HKCategoryTypeIdentifierSleepAnalysis"
  ):
      start, end = child.attrib["startDate"], child.attrib["endDate"]
      location = child[0].attrib["value"]

      # Filter date
      if start < "2022-07-11 00:00:00 +0900":
          continue

      row = [start, end, location]

      print(row)
      writer.writerow(row)

# %% [markdown]
# Convert extracted csv into flatten list.

# %%
from datetime import datetime
from zoneinfo import ZoneInfo

import csv

TIME_FORMAT = "%Y-%m-%d %H:%M:%S"

# Initialize time
# Data is collected from 2022-07-11.
init_time = datetime(2022, 7, 11)
curr_time = datetime.today()

# Initialize sleep list
sleep_list = [0 for _ in range((curr_time - init_time).days)] * 48

with open("data-sleep.csv", "r", encoding="utf-8-sig") as fp:
reader = csv.reader(fp)

for start, end, region in reader:
  start_time = datetime.strptime(start[:-6], TIME_FORMAT)
  end_time = datetime.strptime(end[:-6], TIME_FORMAT)

  # Get timezone info
  try:
      tz_curr = ZoneInfo(region)
  except:
      continue

  start_time = start_time.astimezone(tz_curr).replace(tzinfo=None)
  end_time = end_time.astimezone(tz_curr).replace(tzinfo=None)

  start_delta = start_time - init_time
  end_delta = end_time - init_time

  start_index = (start_delta.days * 48) + (start_delta.seconds // 1800)
  end_index = (end_delta.days * 48) + (end_delta.seconds // 1800)

  # Skip counting if time is before time_init
  if start_index < 0:
      continue

  start_value = (1800 - (start_delta.seconds % 1800)) // 60
  end_value = (end_delta.seconds % 1800) // 60

  for index in range(start_index, end_index + 1):
      if index == start_index:
          sleep_list[index] = start_value
      elif index == end_index:
          sleep_list[index] = end_value
      else:
          sleep_list[index] = 30

print(sleep_list)

# %% [markdown]
# Export sleep list into text

# %%
with open("result-sleep.txt", "w", encoding="utf-8") as fp:
for sleep in sleep_list:
  fp.write(f"{sleep}\n")

# %% [markdown]
# Export sleep list into csv (deprecated maybe...)

# %%
from datetime import datetime, timedelta

init_time = datetime(2022, 7, 11)

with open("result-sleep.csv", "w", encoding="utf-8-sig", newline="") as fp:
writer = csv.writer(fp)

for day in range(len(sleep_list) // 48):
  index = day * 48
  date = datetime.strftime(init_time + timedelta(day), "%Y-%m-%d")
  row = [date, *sleep_list[index : index + 48]]

  writer.writerow(row)

# %% [markdown]
# There is no timezone data in walk data.
# Let's get data from hand-made csv file!

# %%
import csv


def get_region(date: str):
  with open("data-region.csv", "r", encoding="utf-8-sig") as fp:
      reader = csv.reader(fp)
      result = ""

      for bound, region in reader:
          if date >= bound:
              result = region
          else:
              break

  return result

# %% [markdown]
# Let's collect sleep data from xml data!

# %%
import xml.etree.ElementTree as ET
import csv

tree = ET.parse("../export.xml")
root = tree.getroot()

with open("data-walk.csv", "w", encoding="utf-8-sig", newline="") as fp:
  writer = csv.writer(fp)
  for child in root:
      if (
          child.tag == "Record"
          and child.attrib["type"] == "HKQuantityTypeIdentifierStepCount"
      ):
          start = child.attrib["startDate"]
          end = child.attrib["endDate"]
          steps = child.attrib["value"]
          region = get_region(start)

          # Filter date
          if start < "2022-07-11 00:00:00 +0900":
              continue

          row = [start, end, steps, region]

          print(row)
          writer.writerow(row)

# %% [markdown]
# Convert extracted csv into flatten list.

# %%
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo
import csv

TIME_FORMAT = "%Y-%m-%d %H:%M:%S"

# Initialize time and result list
init_time = datetime(2022, 7, 11)
curr_time = datetime.today()

# Initialize walk list
walk_list = [0 for _ in range((curr_time - init_time).days)] * 48

with open("data-walk.csv", "r", encoding="utf-8-sig") as fp:
  reader = csv.reader(fp)

  for start, end, steps, region in reader:
      start_time = datetime.strptime(start[:-6], TIME_FORMAT)
      end_time = datetime.strptime(end[:-6], TIME_FORMAT)

      # Get timezone info
      try:
          tz_curr = ZoneInfo(region)
      except:
          continue
      
      start_time = start_time.astimezone(tz_curr).replace(tzinfo=None)
      end_time = end_time.astimezone(tz_curr).replace(tzinfo=None)

      start_delta = start_time - init_time
      end_delta = end_time - init_time

      start_index = (start_delta.days * 48) + (start_delta.seconds // 1800)
      end_index = (end_delta.days * 48) + (end_delta.seconds // 1800)

      start_value = (1800 - (start_delta.seconds % 1800)) // 60
      end_value = (end_delta.seconds % 1800) // 60

      steps = int(steps) // (end_index - start_index + 1)

      for index in range(start_index, end_index + 1):
          walk_list[index] += steps

print(walk_list)

# %% [markdown]
# Export sleep list into text

# %%
with open("result-walk.txt", "w", encoding="utf-8") as fp:
  for walk in walk_list:
      fp.write(f"{walk}\n")

# %% [markdown]
# Export sleep list into csv (deprecated maybe...)

# %%
from datetime import datetime, timedelta

with open("result-walk.csv", "w", encoding="utf-8-sig", newline="") as fp:
  writer = csv.writer(fp)

  for day in range(len(walk_list) // 48):
      index = day * 48
      date = datetime.strftime(init_time + timedelta(day), "%Y-%m-%d")
      row = [date, *walk_list[index : index + 48]]

      writer.writerow(row)

# %% [markdown]
# ### Merge sleep data with walking data
# 
# Ah, engineering thing!

# %% [markdown]
# Collect sleep and walk data from text result.

# %%
# Start data from 22:00
sleep_list = [0, 0, 0, 0]
walk_list = [0, 0, 0, 0]

with open("result-sleep.txt", "r", encoding="utf-8") as fp:
  sleep_list += [int(line) for line in fp.readlines()]

with open("result-walk.txt", "r", encoding="utf-8") as fp:
  walk_list += [int(line) for line in fp.readlines()]

print(sleep_list)
print(len(sleep_list))
print(walk_list)
print(len(walk_list))

# %% [markdown]
# Zip two data into tuple and apply dynamic daylength.
# 
# Q. Is making element as tuple is really a good idea?

# %%
sleepwalk_data = list(zip(sleep_list, walk_list))
sleepwalk_list = []

for day in range(len(sleepwalk_data) // 48):
  index = day * 48
  buffer = sleepwalk_data[index : index + 48]

  # Append data until the sleeping time occurs.
  # It is guaranteed that `index + 48` is not a sleeping time.
  for sleep, walk in sleepwalk_data[index + 48 : index + 96]:
      if sleep > 0:
          break

      # Add data to buffer
      buffer.append((sleep, walk))

  # Add buffer to result list
  sleepwalk_list.append(buffer)

_ = [print(sleepwalk) for sleepwalk in sleepwalk_list]

# %% [markdown]
# Export sleep list into json.

# %%
from datetime import datetime, timedelta
import json

# Set initial time
init_time = datetime(2022, 7, 11)

# Convert sleepwalk list into json object
json_obj = {}

for day, sleepwalk in enumerate(sleepwalk_list):
  date = datetime.strftime(init_time + timedelta(day), "%Y-%m-%d")
  json_obj[date] = [[sleep, walk] for sleep, walk in sleepwalk]

# Print result log
for k, v in json_obj.items():
  print(k, v)

# Save json object as json file
with open("result-merge.json", "w") as fp:
  json.dump(json_obj, fp)

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.6.0/p5.js"></script>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.11.1/addons/p5.sound.min.js"></script>
  <script src="https://unpkg.com/p5.js-svg@1.5.0"></script>
  <link rel="stylesheet" type="text/css" href="style.css">
  <meta charset="utf-8" />

</head>
<body>
  <main>
  </main>
  <script src="sketch.js"></script>
</body>
</html>

// sketch.js
function preload() {
  dataJSON = loadJSON("result-merge.json");
}

function setup() {
  createCanvas(1600, 5300, SVG);
  noLoop();
}

function draw() {
  //this makes sure there is content in the data
  if (dataJSON) {
      let data = dataJSON.data;
      print(data.length)
      for (let i = 0; i < data.length; i++) {
          let datum = data[i];
          let date = datum.date;
          let sleepwalk = datum.sleepwalk;

          // Add label
          if (i % 7 == 0) {
              fill(0, 0, 0);
              noStroke();
              text(date, 10, 40 * Math.floor(i / 7) + 55);
          }

          for (let j = 0; j < sleepwalk.length; j++) {
              // Unpack sleep and walk value
              let sleep = sleepwalk[j][0];
              let walk = sleepwalk[j][1];

              // Visualize sleep
              let sleep_size = 5;
              let sleep_alpha = map(sleep, 0, 30, 0, 255 * 0.14)

              fill(121, 166, 205, sleep_alpha);
              stroke(121, 166, 205, sleep_alpha * 4);
              let x_sleep = 80 + 10 * j;
              let y_sleep = 50 + 40 * Math.floor(i / 7) - sleep_size / 2;
              square(x_sleep, y_sleep, sleep_size);

              // Visualize walk
              let walk_size = map(walk, 0, 3000, 0, 30);
              let walk_offset = random(-4, 4);
            
              if (walk_size) {
                  fill(231, 95, 65, 255 * 0.14);
                  stroke(231, 95, 65, 255 * 0.56);
              } else {
                  noFill();
                  noStroke();
              }
              
              let x_walk = 80 + 10 * j;
              let y_walk = 50 + 40 * Math.floor(i / 7) + walk_offset - walk_size / 2;
              square(x_walk, y_walk, walk_size);
          }
      }
  }
}

• 주 단위로 묶었는데도 데이터가 너무 많다 (125주, 총 877일).데이터를 줄이거나, 관전 포인트에서 본 것처럼 기간 별로 카테고리화해야 할 것 같다.
• 범례도 디자인틱(??)하게 줘야 한다. (밤을 새니 어휘가 생각이 나지 않습니다)
• 눈에 보여지는 데이터가 어떤 정보를 나타내는지 주석을 달아야 한다.
• 이제 레이아웃을 고민해야 한다 (제일 어려운 것)
• 최적화: 지금 저 그래픽을 그대로 내보내면 오브젝트가 너무 많다.
  보이지는 않지만, 여전히 그려진 오브젝트가 많고, 겹쳐 있는 녀석들도 많아서 그렇다.(특히 수면 데이터 파트..컴퓨터가 울부짖습니다…)
  데이터셋을 일 단위에서 주 단위로 묶고, 보이지 않는 오브젝트는 그리지 않게 최적화.

// sketch.js
function preload() {
dataJSON = loadJSON("assets/result-merge.json");
font = loadFont("assets/Pretendard-Bold.otf");
}

function setup() {
createCanvas(1600, 5300, SVG);  // Create canvas
noStroke();                     // We don't use stroke
noLoop();                       // Iterate only once
}

function draw() {
//this makes sure there is content in the data
if (dataJSON) {
let data = dataJSON.data;
let weekly_sleep = new Array(96).fill(0);

for (let i = 0; i < data.length; i++) {
let datum = data[i];
let date = datum.date;
let sleepwalk = datum.sleepwalk;

// Work to do for the first weekday
if (i % 7 == 0) {
  // Add title for row
  fill(0, 0, 0);
  textSize(10);
  textFont(font);
  text(date, 10, 40 * Math.floor(i / 7) + 55);

  // Reset weekly sleep array
  weekly_sleep = new Array(96).fill(0);
}

// Iterate blocks in daily sleepwalk data
for (let j = 0; j < sleepwalk.length; j++) {
  // Unpack sleep and walk value
  let sleep = sleepwalk[j][0];
  let walk = sleepwalk[j][1];

  // Append sleep data into weekly sleep
  weekly_sleep[j] += sleep;

  // Calculate pivot coordinate
  let x = 80 + 10 * j;
  let y = 50 + 40 * Math.floor(i / 7);

  // Visualize walk
  let walk_size = map(walk, 0, 3000, 0, 30);
  let walk_offset = random(-4, 4);

  if (walk_size) {
    let x_walk = x - walk_size / 2;
    let y_walk = y + walk_offset - walk_size / 2;

    fill(220, 70, 20, 255 * 0.25);
    square(x_walk, y_walk, walk_size);
  }
}

// Work to do for the last weekday
if (i % 7 == 6) {
  // Draw weekly sleep data
  for (let j = 0; j < 96; j++) {
    // Unpack sleep value from weekly sleep
    let sleep = weekly_sleep[j];

    // Calculate pivot coordinate
    let x = 80 + 10 * j;
    let y = 50 + 40 * Math.floor(i / 7);

    // Visualize sleep
    let sleep_size = 8;
    let sleep_alpha = map(sleep, 0, 210, 0, 255)

    if (sleep_alpha) {
      let x_sleep = x - sleep_size / 2;
      let y_sleep = y - sleep_size / 2;

      fill(25, 50, 75, sleep_alpha);
      square(x_sleep, y_sleep, sleep_size);
    }
  }
}
}
}
}

// sketch.js
function preload() {
dataJSON = loadJSON("assets/result-merge.json");
font = loadFont("assets/Pretendard-Bold.otf");
}

function setup() {
createCanvas(1300, 3900, SVG);  // Create canvas
noStroke();                     // We don't use stroke
noLoop();                       // Iterate only once
}

function draw() {
//this makes sure there is content in the data
if (dataJSON) {
let data = dataJSON.data;
let weekly_sleep = new Array(96).fill(0);

for (let i = 0; i < 630; i++) {  // 7 days * 90 weeks
let datum = data[i];
let date = datum.date;
let sleepwalk = datum.sleepwalk;

// Work to do for the first weekday
if (i % 7 == 0) {
  // Add title for row
  fill(0, 0, 0);
  textSize(10);
  textFont(font);
  text(date, 50, 205 + 30 * Math.floor(i / 7));

  // Reset weekly sleep array
  weekly_sleep = new Array(96).fill(0);
}

// Iterate blocks in daily sleepwalk data
for (let j = 0; j < sleepwalk.length; j++) {
  // Unpack sleep and walk value
  let sleep = sleepwalk[j][0];
  let walk = sleepwalk[j][1];

  // Append sleep data into weekly sleep
  weekly_sleep[j] += sleep;

  // Calculate pivot coordinate
  let x = 120 + 15 * j;
  let y = 200 + 30 * Math.floor(i / 7);

  // Visualize walk
  let walk_size = map(walk, 0, 3000, 0, 30);
  let walk_offset = random(-4, 4);

  if (walk_size) {
    let x_walk = x - walk_size / 2;
    let y_walk = y + walk_offset - walk_size / 2;

    fill(220, 70, 20, 255 * 0.25);
    square(x_walk, y_walk, walk_size);
  }
}

// Work to do for the last weekday
if (i % 7 == 6) {
  // Draw weekly sleep data
  for (let j = 0; j < 96; j++) {
    // Unpack sleep value from weekly sleep
    let sleep = weekly_sleep[j];

    // Calculate pivot coordinate
    let x = 120 + 15 * j;
    let y = 200 + 30 * Math.floor(i / 7);

    // Visualize sleep
    let sleep_size = 12;
    let sleep_alpha = map(sleep, 0, 210, 0, 255)

    if (sleep_alpha) {
      let x_sleep = x - sleep_size / 2;
      let y_sleep = y - sleep_size / 2;

      fill(25, 50, 75, sleep_alpha);
      square(x_sleep, y_sleep, sleep_size);
    }
  }
}
}
}
}