WeiNote

yyrcd

web

TA

pl

2020

May 23

sudo netstat -tunlp
  • -t - Show TCP ports.
  • -u - Show UDP ports.
  • -n - Show numerical addresses instead of resolving hosts.
  • -l - Show only listening ports.
  • -p - Show the PID and name of the listener’s process. This information is shown only if you run the command as root or sudo user.

2020

May 19

Screen - linux

screen -ls
screen -L       # output to a file named as screenlog.#
screen -S name  # give the session a name
screen -r [session]

2020

May 15

Git Large File Storage (git-lfs)

ubuntu

sudo apt-get install git-lfs
git lfs install

mac

brew install git-lfs
git lfs install

usage

git lfs track "*.psd"
git add .gitattributes
git add file.psd
git commit -m "Add design file"
git push origin master
git lfs ls-files
git lfs pull --all

Reference: Git Large File Storage

2020

May 10

2020

Apr 27

2020

Apr 26

Git Config

git config --global user.name "richard"
git config --global user.email "yueyericardo@gmail.com"
git config --global color.ui true 

git config --global alias.lg "log --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit"
git config --global alias.last 'log -1'
git config --global alias.unstage 'reset HEAD'
git config --global alias.st 'status'
git config --global core.quotepath false  # Chinese

2020

Apr 25

Sass to CSS

node-sass 的安装和使用

node-sass --watch ~/dev/WeiNote/public/css/wiki.scss ~/dev/WeiNote/public/css/wiki.css

2020

Apr 24

Download m3u8

ffmpeg -user_agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/601.7.8 (KHTML, like Gecko) Version/9.1.3 Safari/537.86.7" \
-i https://link.com/playlist.m3u8 -c copy download.mkv

2020

Apr 07

Format an USB on mac

sudo diskutil eraseDisk JHFS+ richard disk5

2020

Mar 31

find

find . -iname vectoradd
./0_Simple/vectorAdd/vectorAdd
find . -iname *.cu
./3_Imaging/SobelFilter/SobelFilter_kernels.cu
./3_Imaging/postProcessGL/postProcessGL.cu
....

Ref: Linux find命令 | 菜鸟教程

2020

Mar 13

chmod

chmod 777 file
x: 1
w: 2
r: 4

2020

Mar 10

conda rollback to previous version

conda list --revisions
conda install --revision=14

2020

Mar 03

How to upgrade from 18.10 to 19.10, using the command line? - Ask Ubuntu When run do-release-upgrade on the 18.10 system. This will give you an error about being unsupported.

2020

Mar 02

2020

Feb 08

mac sshd config

PermitRootLogin no
PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
UsePAM no  # important

then restart

sudo launchctl stop com.openssh.sshd
sudo launchctl start com.openssh.sshd

2020

Feb 05

root密码忘记了怎么半?

linux安装盘,比如ubuntu安装盘

mount -t auto /dev/系统盘
cd /etc/
nano passwd
# remove x in next line
root:x:0:0:root:/root:/bin/bash

然后就可以tty以root登陆,不需要密码 然后,passwd重制密码

2020

Feb 02

quick access a file in server

python -m http.server 8080

2019

2019

Dec 25

Linux – top命令详解 | | Bruce's Blog 1 show all cpus t change cpu display m change momery display E KiB MiB GiB W save current config

2019

Dec 21

Dock-compose /mnt/hdd/richard/mongo/mongo local folder to mount

mongo: 
  volumes:
    - /mnt/hdd/richard/mongo/mongo:/data/db

2019

Dec 20

Git stash

git stash
git stash list
git stash pop  #(recover)

2019

Dec 12

Usage of dcp

cd ~/
mkdir apps
cd apps
wget https://yyrcd-1256568788.cos.na-siliconvalley.myqcloud.com/yyrcd/2019-12-12-dcp
mv 2019-12-12-dcp dcp
~/apps/dcp 

2019

Nov 20

Use python in bash script

Simple case:

for i in {0..5}
do
    file=$(python -c "a='wavefunction{:02d}'.format($i); print(a)")
    echo $file
    sleep 0.5
done
#wavefunction00
#...
#wavefunction05
python -c "print('some')"
some  #output
a=$(python -c "a='I am {}'.format('richard'); print(a)")
echo $a
I am richard   #output
a=$(python -c "print('some')" 2>&1)  # 2>&1的意思是python的stderr输出重定向到stdout
echo $a
some   #output
for i in {0..29}
do
    echo $i
    python -c "import os; os.remove('input'); a = 'set nat = \'/ufrc/alberto.perezant/arup.mondal/project/nmr/1st_trial/metaclust/ExperimentalStructures/N0968s2_leap.pdb\'\ntrajin follow.{:02d}.dcd\nparm \$nat\nreference \$nat parm \$nat\nrmsd reference :1-115@CA out rmsd_walker/rmsd.{}.txt\ngo\nclear trajin'.format($i,$i); print($a); file=open('input', 'w+'); file.write(a); file.close()"
    cpptraj -p ../template.pdb -i input
done
set nat = '/ufrc/alberto.perezant/arup.mondal/project/nmr/1st_trial/metaclust/ExperimentalStructures/N0968s2_leap.pdb'
trajin follow.29.dcd
parm $nat
reference $nat parm $nat
rmsd reference :1-115@CA out rmsd_walker/rmsd.29.txt
go
clear trajin
~

2019

Nov 16

unzip alternative

ditto -V -x -k --sequesterRsrc --rsrc file.zip dir/

2019

Nov 07

Check if is SSD drive

lsblk -d -o name,rota

2019

Oct 30

docker

part 1 orientation

docker
docker container --help

docker --version  # Display Docker version and info
docker version
docker info

docker run hello-world  # Execute Docker image

docker image ls  # List Docker images

docker container ls  #List Docker containers (running, all, all in quiet mode)
docker container ls --all
docker container ls -aq

part 2 containers

docker build -t friendlyhello .  # Create image using this directory's Dockerfile
docker run -p 4000:80 friendlyhello  # Run "friendlyname" mapping port 4000 to 80
docker run -d -p 4000:80 friendlyhello         # Same thing, but in detached mode
docker container ls                                # List all running containers
docker container ls -a             # List all containers, even those not running
docker container stop <hash>           # Gracefully stop the specified container
docker container kill <hash>         # Force shutdown of the specified container
docker container rm <hash>        # Remove specified container from this machine
docker container rm $(docker container ls -a -q)         # Remove all containers
docker image ls -a                             # List all images on this machine
docker image rm <image id>            # Remove specified image from this machine
docker image rm $(docker image ls -a -q)   # Remove all images from this machine
docker login             # Log in this CLI session using your Docker credentials
docker tag <image> username/repository:tag  # Tag <image> for upload to registry
docker push username/repository:tag            # Upload tagged image to registry
docker run username/repository:tag                   # Run image from a registry

part 3 services

docker stack ls                                            # List stacks or apps
docker stack deploy -c <composefile> <appname>  # Run the specified Compose file
docker service ls                 # List running services associated with an app
docker service ps <service>                  # List tasks associated with an app
docker inspect <task or container>                   # Inspect task or container
docker container ls -q                                      # List container IDs
docker stack rm <appname>                             # Tear down an application
docker swarm leave --force      # Take down a single node swarm from the manager

2019

Oct 28

Set power supply for GPU

sudo nvidia-smi -q -d POWER  # check the current limit
sudo nvidia-smi -i 2 -pl 250  # set power limit of gpu3 to 250W
sudo nvidia-smi -pm 1  # set to persistent mode (no need to change everytime start?)

2019

Oct 22

Slurm, there is resource, but why my job is still pending?

There might be one resource job is also pending. Then you submitted another job, though resources are enough to run your job. And your job is also pending, the reason is priority.

Whether slurm will run your job will depend on how long your priority job needs to run. for example: This job might take few hours, then slurm might determin it could run. If it will take days, might not, because the resource job came first, and he will needs to wait days to get his job started.

2019

Oct 04

Update Slurm after Replace GPU

If chang gpu on mordor or isengard, both of them needs to be updated

sudo vim /etc/slurm-llnl/slurm.conf
sudo vim /etc/slurm-llnl/gres.conf

Then restart

sudo /etc/init.d/slurmd restart
sudo /etc/init.d/slurmctld restart

2019

Sep 19

Ubuntu add user

sudo adduser username
sudo usermod -aG sudo username

2019

Sep 04

Slurm: Kill task failed

And state=MIXED+DRAIN

  • log file

    richard@moria~> sudo cat /var/log/slurm-llnl/slurmctld.log
    [2019-09-04T15:16:31.313] backfill: Started JobID=2076 in queue on moria
    [2019-09-04T15:36:52.152] _slurm_rpc_submit_batch_job: JobId=2077 InitPrio=4294899834 usec=645
    [2019-09-04T15:37:01.327] backfill: Started JobID=2077 in queue on moria
    [2019-09-04T15:38:41.041] _slurm_rpc_kill_job: REQUEST_KILL_JOB job 2077 uid 1026
    [2019-09-04T15:38:43.734] _slurm_rpc_submit_batch_job: JobId=2078 InitPrio=4294899833 usec=648
    [2019-09-04T15:39:34.232] Resending TERMINATE_JOB request JobId=2077 Nodelist=moria
    [2019-09-04T15:39:42.055] error: slurmd error running JobId=2077 on node(s)=moria: Kill task failed
    [2019-09-04T15:39:42.055] drain_nodes: node moria state set to DRAIN
    [2019-09-04T15:39:42.081] cleanup_completing: job 2077 completion process took 61 seconds
  • nodes information

    richard@moria~> scontrol show nodes
    NodeName=moria Arch=x86_64 CoresPerSocket=8
     CPUAlloc=4 CPUErr=0 CPUTot=16 CPULoad=4.34
     AvailableFeatures=(null)
     ActiveFeatures=(null)
     Gres=gpu:gtx1080ti:2,gpu:titanv:3,gpu:v100:1,gpu:gp100:2
     NodeAddr=moria NodeHostName=moria Version=17.11
     OS=Linux 4.15.0-45-generic #48-Ubuntu SMP Tue Jan 29 16:28:13 UTC 2019 
     RealMemory=120000 AllocMem=0 FreeMem=1023 Sockets=2 Boards=1
     State=MIXED+DRAIN ThreadsPerCore=1 TmpDisk=0 Weight=1 Owner=N/A MCS_label=N/A
     Partitions=queue 
     BootTime=2019-02-03T15:36:08 SlurmdStartTime=2019-02-13T18:00:18
     CfgTRES=cpu=16,mem=120000M,billing=16
     AllocTRES=cpu=4
     CapWatts=n/a
     CurrentWatts=0 LowestJoules=0 ConsumedJoules=0
     ExtSensorsJoules=n/s ExtSensorsWatts=0 ExtSensorsTemp=n/s
     Reason=batch job complete failure [slurm@2019-09-04T15:39:42]
  • how to fix

    richard@moria~> scontrol update nodename=moria state=resume
  • For more information

  • show cpu information

    richard@moria~> sinfo -O cpusstate
    CPUS(A/I/O/T)(allocated/idle/other/total)
    5/11/0/16 

2019

Aug 18

git undo a commit

git reset --hard HEAD^
git push origin +branch_name  # force push 

2019

Aug 14

create a restricted guest user on linux

# root
useradd -s /bin/rbash -d /home/guest guest
mkdir /home/guest
chown guest:guest /home/guest
passwd guest

At this point, guest user still can access all the commands which are in $PATH. At least you need to remove all shell from $PATH. E.G. bash zsh

change guest shell to specific command

# write command to .sh file
sudo chsh -s /home/guest/guest-shell.sh guest

2019

Aug 07

netdata

Web API

Webfile location: /opt/netdata/netdata-web-files/my.html

API usage: http://pc.moyi.yyrcd.com/api/v1/chart?chart=system.ram

API for shell

eval "$(curl -s 'http://localhost:19999/api/v1/allmetrics')"

# GPU RAM
NETDATA_NVIDIA_SMI_GPU0_MEM_USAGE_USED
# GPU UTIL
NETDATA_NVIDIA_SMI_GPU0_GPU_UTILIZATION_UTILIZATION

# CPU RAM
NETDATA_SYSTEM_RAM_USED
# CPU UTIL
NETDATA_SYSTEM_CPU_VISIBLETOTAL

2019

Aug 06

NFS setting

server:

sudo apt install nfs-kernel-server
sudo vim /etc/exports

client:

sudo apt install nfs-common
sudo vim /etc/fstab

if client is mac

use vifs instead of /etc/fstab, more detail

useful commands

showmount -e : Shows the available shares on your local machine
showmount -e <server-ip or hostname>: Lists the available shares at the remote server
showmount -d : Lists all the sub directories
exportfs -v : Displays a list of shares files and options on a server
exportfs -a : Exports all shares listed in /etc/exports, or given name
exportfs -u : Unexports all shares listed in /etc/exports, or given name
exportfs -r : Refresh the server’s list after modifying /etc/exports

2019

Jul 02

git

所有更改都add上

git add -u .

remove from git, but keep in local

git rm --cached mylogfile.log 
git rm --cached -r directory

2019

Jul 01

常用network工具

ip route
default via 10.30.160.1 dev enp3s0 proto dhcp metric 100 
10.30.160.0/20 dev enp3s0 proto kernel scope link src 10.30.171.30 metric 100 
169.254.0.0/16 dev enp3s0 scope link metric 1000 

ip route get 10.241.124.148
10.241.124.148 via 10.136.0.1 dev en0  src 10.136.196.12

2019

Jun 24

linux系统硬件信息

cat /proc/cpuinfo
lspci
lscpu
lshw # 最全

linux安装ssd硬盘

vim /etc/fstab  #安装硬盘上之后,mount到某文件夹;每次开机都会自动mount

Show RAM information

$ sudo lshw -short -C memory
H/W path      Device     Class          Description
===================================================
/0/0                     memory         64KiB BIOS
/0/3c                    memory         24GiB System Memory
/0/3c/0                  memory         8GiB DIMM DDR4 Synchronous Unbuffered (Unregistered) 2133 MHz (0.5 ns)
/0/3c/1                  memory         [empty]
/0/3c/2                  memory         16GiB DIMM DDR4 Synchronous Unbuffered (Unregistered) 2133 MHz (0.5 ns)
/0/3c/3                  memory         [empty]
/0/42                    memory         256KiB L1 cache
/0/43                    memory         1MiB L2 cache
/0/44                    memory         6MiB L3 cache
/0/100/1f.2              memory         Memory controller

2019

Jun 10

GPU status

watch -n 1 -d nvidia-smi 

2019

Jun 10

保持fork之后的项目和上游同步

git remote add upstream https://github.com/staticblog/staticblog.github.io.git
git remote -v
git fetch upstream
git checkout master
git merge upstream/master
origin  git@github.com:ibrother/staticblog.github.io.git (fetch)
origin  git@github.com:ibrother/staticblog.github.io.git (push)
upstream        https://github.com/staticblog/staticblog.github.io.git (fetch)
upstream        https://github.com/staticblog/staticblog.github.io.git (push)

2019

Jun 07

jupyter

jupyter notebook password
jupyter notebook --generate-config # ip port open_browser
conda config --set changeps1 False #terminal是否显示env名字

2019

Jun 07

git reset everything

git checkout -- .

2019

Jun 07

vim Comment multiple lines https://stackoverflow.com/a/1676690/9581569

2019

Jun 07

nohup command &

nohup scp -P 2022 confs_gdb01-06_red03-07_cv_1_chkpts.tar.gz \
ly.yyrcd.com:/mnt/data/richard/ &

jobs
kill %1

2019

Jun 07

vim copy cut paste

y copy
d cut
p paste