这篇博客介绍了一些日常收集整理资料。

常见的日常积累小技巧

1. mac 电脑外接显示屏 对于屏幕分辨率的问题，其中较好的一个仓库软件叫做betterdisaplay，可以用来调整刷新赫兹以及分辨率，但是要注意下载符合mac的系统版本的软件； https://github.com/waydabber/BetterDisplay/releases?page=6

2. adobe ps ：https://www.cnblogs.com/FRIM/p/18130753

3. 绘图的时候图例位置参数设置

    # 紧凑型图例
    handles, labels = plt.gca().get_legend_handles_labels()
    plt.legend(handles[:35], labels[:35], loc='upper right', 
              ncol=2, columnspacing=0.8, handletextpad=0.5)
       
    # 自动检测最佳位置
    # plt.legend(
    #     loc='best',
    #     bbox_to_anchor=(1, 0.5),  # 右侧垂直居中
    #     borderaxespad=0.5,
    #     framealpha=0.9
    # )
       
    # plt.legend(
    # loc='upper center',
    # bbox_to_anchor=(0.5, -0.15),  # 向下偏移15%
    # ncol=2,
    # frameon=True,
    # shadow=True,
    # fancybox=True
    # )
   

google colab 时刻触发脚本

function ConnectButton(){
    console.log("Connect pushed");
    document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click()
}
setInterval(ConnectButton,60000);

2. 后台进程管理方案

• 使用nohup持久运行 nohup python -u run_script.py > script.log 2>&1 &

• 参数说明

  nohup：忽略挂断信号
  -u（Python参数）：禁用输出缓冲
  > script.log：标准输出重定向
  2>&1：错误输出合并
  &：后台运行
  

• 验证进程状态

  查看进程列表
  ps aux | grep "python run_script.py"
  实时监控日志
  tail -f script.log
  

• 终止进程

  #优雅终止
  pkill -f "python run_script.py"
  #强制终止（无响应时）
  kill -9 1708
  

• 高级管理方案（生产环境推荐） 使用systemd服务 创建服务文件： sudo nano /etc/systemd/model.service

• 服务内容：

  [Unit]
  Description=Model Training Service
  [Service]
  User=root
  WorkingDirectory=/root/PDFormer
  ExecStart=/root/miniconda3/envs/base/bin/python run_script.py
  Restart=always
  StandardOutput=file:/var/log/model.log
  StandardError=file:/var/log/model_error.log
  [Install]
  WantedBy=multi-user.target
  

• 启用服务：

  sudo systemctl daemon-reload
  sudo systemctl enable model
  sudo systemctl start model
  

• 使用tmux会话管理

  安装tmux
  sudo apt install tmux -y
  创建会话
  tmux new -s model_train
  在会话中运行
  python run_script.py
  分离会话：
  Ctrl+B → D
  重连会话：
  tmux attach -t model_train
  要打开现有的tmux会话"python_session"，请使用以下命令：
  tmux attach -t python_session
  

markdown

• Markdown中常用的快捷键

    Ctrl 0 到 Ctrl 6： 普通文本、一级文本~六级文本
    Ctrl B： 加粗；加粗测试
    Ctrl I： 斜体；斜体测试
    Ctrl U： 下划线；下划线测试
    Shift Alt 5： 删除线；删除线测试
    Shift Ctrl ~： 行内代码块；行内代码块测试
    Ctrl K： 超链接，[超链接测试；欢迎点一个大大的关注！！！](《LL》 - 博客园 (cnblogs.com))；还支持文章内锚点，按Ctrl 键点击此处 👉第一节
    Ctrl T： 表格，支持拖拽移动、网页端表格复制转换
    Ctrl Shift Q： 引用；
    Shift Ctrl I： 插入图片；
    Shift Ctrl M： 公式块；
    [ ]： 任务列表(可勾选的序列)注意每一个符号之间都有空格
    <sup> 内容 </sup>： 上标；上标测试
    <sub> 内容 </sub>： 下标；  下标测试  
    :smile:： 😄
    [toc]： 展示目录
    Ctrl l： 选中一行
    Ctrl d： 选中内容/单词
    Ctrl home： 跳转到文章开头
    Ctrl end： 跳转到文章结尾
    Ctrl f： 搜索
    Ctrl h： 替换

• 美剧 Home Watch TV Show Hacks - Season 1 - Episode 1
• 书籍

  http://38.147.170.240:5959/c1/baidupan/222222/%E5%85%B6%E5%AE%83/EPUB
  本站网址
  http://103.74.192.62:1234
  http://103.74.192.62
  主站http://www.https.ng
  http://38.147.170.240 http://38.147.170.240:5959
  http://45.145.228.151
  http://45.145.228.171
  http://193.134.211.102:1234 http://193.134.211.108:1234
  前面不要加httpS，是http
  

  ---

  traffic 学习 This list can be considered outdated. For a more up-to-date list, check: https://github.com/lixus7/Time-Series-Works-Conferences

Traffic Prediction

Traffic prediction is the task of predicting future traffic measurements (e.g. volume, speed, etc.) in a road network (graph), using historical data (timeseries).

Things are usually better defined through exclusions, so here are similar things that I do not include:

• NYC taxi and bike (and other similar datsets, like uber), are not included, because they tend to be represented as a grid, not a graph.

• Predicting human mobility, either indoors, or through checking-in in Point of Interest (POI), or through a transport network.

• Predicting trajectory.

• Predicting the movement of individual cars through sensors for the purpose of self-driving car.

• Traffic data imputations.

• Traffic anomaly detections.

The papers are haphazardly selected.

Summary

A tabular summary of paper and publically available datasets. The paper is reverse chronologically sorted. NO GUARANTEE is made that this table is complete or accurate (please raise an issue if you spot any error).

Performance

NOTES: The experimental setttings may vary. But the common setting is:

• Observation window = 12 timesteps

• Prediction horizon = 1 timesteps

• Prediction window = 12 timesteps

• Metrics = MAE, RMSE, MAPE

• Train, validation, and test splits = 7/1/2 OR 6/2/2

However, there are many caveats:

• Some use different models for different prediction horizon.

• Some use different batch size when testing previous models, as they increase the observation and prediction windows from previous studies, and have difficulties fitting it on GPU using the same batch size.

• Regarding adjacency matrix, some derive it using Gaussian RBF from the coordinates, some use the actual connectivity, some simply learn it, and some use combinations.

• Some might also add more context, such as time of day, or day of the week, or weather.

• DeepGLO in particular, since it is treating it as a multi-channel timeseries without the spatial information, use rolling validation,

• Many different treatment of missing datasets, from exclusion to imputations.

Dataset

Publically available datasets and where to find them.

• METR-LA DCRNN Google Drive; DCRNN Baidu; Sensor coordinates and adjacency matrix, also from DCRNN

• California department of transportation (Caltrans) Performance Measurement System (PeMS). The website is: http://pems.dot.ca.gov/. From the website: The traffic data displayed on the map is collected in real-time from over 39,000 individual detectors. These sensors span the freeway system across all major metropolitan areas of the State of California

  • PeMS-BAY DCRNN Google Drive; DCRNN Baidu

  Sensor coordinates and adjacency matrix, DCRNN github

  • PeMS-D7(M) PKUAI26 STGCN Github

  • PeMS-D7(L)

  • PeMS-04 ATSGCN Github; Baidu with code: “p72z”; From Davidham3 Github STSGCN

  • PeMS-08 ATSGCN github; Baidu with code: “p72z”; From Davidham3 github STSGCN

  • PeMS-03 Baidu with code: “p72z”; From Davidham3 github STSGCN

  • PeMS-07 Baidu with code: “p72z”; From Davidham3 github STSGCN

  • PeMS-SF UCI

  • PeMS-11160 GP-DCRNN github

• LOOP https://github.com/zhiyongc/Seattle-Loop-Data

• Q-Traffic https://github.com/JingqingZ/BaiduTraffic

Baidu, code: ‘umqd’

• RMTMC - MnDOT https://www.d.umn.edu/~tkwon/TMCdata/TMCarchive.html The data in this archive are continuously collected by the Regional Trasportation Management Center (RTMC), a division of Minesotta Deaprtment of Transport (MnDOT) USA.

• OpenITS http://www.openits.cn/openData/index.jhtml

• FHWA https://www.fhwa.dot.gov/policyinformation/tables/tmasdata/

The following datasets are not publically available:

• INRIX https://pdfs.semanticscholar.org/4b9c/9389719caff7409d9f9cee8628aef4e38b3b.pdf

• Beijing

  • BJER4

  • BJF

  • BRF

  • BRF-L

  • W3-715

  • E5-2907

  • NE-BJ https://github.com/tsinghua-fib-lab/Traffic-Benchmark

• Xiamen https://ieeexplore.ieee.org/document/8029849

Also relevant:

• Davidham3 list

Libraries

• LibCity GitHub

• Tsinghua Fib Lab GitHub

• PyTorch Geometric

Paper

The papers are sorted alphabetically based on model name. The citations are based on Google scholar citation.

You can find the bibtex in traffic_prediction.bib (not complete yet)

Things that would be in the table above if I have more time:

Other works

• Multi-Attention Temporal and Graph Convolution Network for Traffic Flow Forecasting PyTorch

• Foreseeing Congestion using LSTM on Urban Traffic Flow Clusters ICSAI 2019 Keras; dataset: CityPulse

• Using LSTM and GRU neural network methods for traffic flow prediction IEEE YAC 2016 Keras; dataset: PeMS but different from everyone else

• A Dynamic Traffic Awareness System for Urban Driving IEEE GreenCom 2019 Keras; dataset: CityPulse

• Inductive Graph Neural Networks for Spatiotemporal Kriging (IGNNK) AAAI 2021 PyTorch dataset: METR-LA, PeMS-BAY, LOOP, NREL, USHCN.

Other works that is not based on a static-spatial-graph of timeseries:

• VLUC

• STDN

• Deep Representation Learning for Trajectory Similarity Computation

• Curb-GAN SIG KDD 2020

• BusTr SIG KDD 2020

• DeepMove WWW 2018

• https://github.com/Alro10/deep-learning-time-series

• https://github.com/henriquejosefaria/CSC

• https://github.com/shakibyzn/Traffic-flow-prediction

• https://deepmind.com/blog/article/traffic-prediction-with-advanced-graph-neural-networks

Other lists:

• Davidham3 list

• https://paperswithcode.com/task/traffic-prediction

• A Survey on Modern Deep Neural Network for Traffic Prediction: Trends, Methods and Challenges IEEE TKDE 2020

• A Comprehensive Survey on Traffic Prediction to be published in IEEE Transactions on Intelligent Transportation Systems.

• A Comprehensive Survey on Graph Neural Networks IEEE Trans. Neural Netw. Learn. Syst. 2020

• A Comprehensive Survey on Geometric Deep Learning IEEE Access 19 Feb 2020.

• Graph Neural Network for Traffic Forecasting: A Survey Expert Systems with Applications GitHub

• BigsCity LibCity: Traffic Prediction Paper Collection In the paper collection, we collected traffic prediction papers published in the recent years (2016-now) on 11 top conferences and journals, namely, AAAI, IJCAI, KDD, CIKM, ICDM, WWW, NIPS, ICLR, SIGSPATIAL, IEEE TKDE and IEEE TITS. In addition, the surveys since 2016 and representative papers mentioned in the surveys are also included.

• Older, pre-ML approaches: On the modeling of traffic and crowds: A survey of models, speculations, and perspectives SIAM 2011

• https://github.com/Knowledge-Precipitation-Tribe/Urban-computing-papers

• https://github.com/jdlc105/Must-read-papers-and-continuous-tracking-on-Graph-Neural-Network-GNN-progress

• DL-Traff: Survey and Benchmark of Deep Learning Models for Urban Traffic Prediction CIKM 2021 GitHub PyTorch

Acknowledgement

• Willianto Aslim