분류 전체보기(75)
-
[논문리뷰] DriveLM:Driving with Graph Visual Question Answering
DriveLM : Driving with Graph Visual Question AnsweringChonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, Hongyang LiECCV 24 -https://arxiv.org/abs/2312.14150 Summary 일반화 부족: 새로운 환경이나 센서 설정에 대한 적응력이 낮음.설명 가능성 부족: 유럽연합(EU) 등에서 요구하는 투명한 의사결정 과정을 제공하지 못함.단계별 사고 과정 결여: 인간 운전자는 **객체 인식(Perception) → 예측(Prediction) → 계획(Planning)**..
2025.02.23 -
torch.cuda.amp.GradScaler
torch.cuda.amp.GradScaler는 PyTorch에서 Mixed Precision Training (혼합 정밀도 학습, FP16) 시 그레디언트 스케일링을 담당하는 클래스이 기능을 사용하면 FP16 연산의 안정성을 높이고 학습 속도를 가속화할 수 있음 scaler = torch.cuda.amp.GradScaler(init_scale=2048, growth_interval=1000)예시 분석 📌 1. init_scale=2048초기 스케일 팩터 (Gradient Scale Factor)를 2048로 설정기본값은 2.0 ** 16 = 65536이지만, 여기서는 2048로 낮게 설정됨FP16 연산 시 작은 그레디언트 값이 언더플로우(Underflow)되는 문제를 방지스케일이 너무 작으면 학습이 ..
2025.02.21 -
[논문리뷰] LoRA: Low-Rank Adaptation of Large Language Models
LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen | MSICLR 22 Contents1. ChatGPT 1-1. GPT-1 1-2. GPT-2 1-3. GPT-3 1-4. GPT-4 1-5. Instruct GPT - [https://jaeha-lee.tistory.com/91] 1-6. RLHF2. LLama 3. Gemini4. DeepSeek5. LLaVA 5-1. LLaVA - https://jaeha-lee.tistory.com..
2025.02.12 -
[논문리뷰] LLaVA : Visual Instruction Tuning
Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023[https://arxiv.org/abs/2304.08485] Contents1. ChatGPT 1-1. GPT-1 1-2. GPT-2 1-3. GPT-3 1-4. GPT-4 1-5. Instruct GPT 1-6. RLHF2. LLama 3. Gemini4. DeepSeek5. LLaVA 5-1. LLaVA - https://jaeha-lee.tistory.com/93 5-2. LLaVA-Next 5-3. LLaVA-NeXT-Interleave6. LoRA SummaryVisual Instruc..
2025.02.12 -
LLM / VLM paper list
LLMChatGPT PapersGPT-1 : Improving Language Understanding by Generative Pre-Training. - Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever [pdf] 2018.6GPT-2 : Language Models are Unsupervised Multitask Learners. - Alec Radford, Jeff Wu, Rewon Child, D. Luan, Dario Amodei, Ilya Sutskeve [pdf] 2019.2GPT-3 : Language Models are Few-Shot Learners. - Tom B. Brown, Benjamin Mann, Nick Ryde..
2025.02.11 -
[논문리뷰] (InstructGPT) Training language models to follow instructions with human feedback 논문 리뷰 요약
Improving Language Understanding by Generative Pre-TrainingInstructGPT - [2203.02155] Training language models to follow instructions with human feedback]Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, OpenAINeurIPS, 2022 - https://arxiv.org/abs/2203.02155 Contents1. ChatGPT 1-1. GPT-1 1-2. GPT-2 1-3. GPT-3 1-4. GPT-4 1-5. Instruct GPT - (Tr..
2025.02.06