Haishan Ye
 |
Associate Professor (Tenure Track) Contactyehaishan@xjtu.edu.cn |
About Me
I am now an Associate Professor at Xi'an JiaoTong University, and also a Research Scientist in SGIT AI Lab, State Grid Corporation of China. I received my Ph.D. on Computer Science at Shanghai Jiaotong University under my supervisor Prof. Zhihua Zhang (Peking University Now). I was a Postdoctoral Researcher in the Hongkong University of Science and Technology (HKUST), working with Prof. Tong Zhang (UICU Now).
My research interests are broadly in the machine learning algorithms and theory. I currently work on various aspects of algorithm optimization and numerical linear algebra. The topics include, but are not limited to,
- Algorithm Optimization: convex opt., stochastic opt., distributed/federated opt., min-max opt., etc. I focus on building more realistic mathematical foundation for solvers on machine learning models and designing faster algorithms.
- Numerical Linear Algebra: sparse matrix decomposition, iterative methods and preconditioning, etc. I am interested in proposing new models or analyses to enhance the theoretical understanding of numerical linear algebra.
- AI Foundation: I am interested in proposing new algorithms and methodologies to pretrain, fine-tune, align and attack large language models.
News
|
1、 I am recruiting self-motivated MPhil and PhD students, and long-term interns for machine learning and artificial intelligence at SGIT AI Lab. We will provide abundant GPU computing resources and generous living subsidies during the internship. In addition, our working hours are flexible. For more information, please click on the link below, and we welcome everyone to join the laboratory. If you are interested, please send your detailed CV to my email. (新加坡A*STAR-国网思机未来人工智能(SGIT AI)联合研究实习岗位开放) 2、Three papers have been accepted to ICML2024. |
Publications (*Corresponding author, † Equal contribution)
Conference Publications
- Dachao Lin, Yuze Han, Haishan Ye*, Zhihua Zhang.
Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysis.
Advances in Neural Information Processing Systems(NeurIPS),2024.
[pdf] - Jun Chen, Haishan Ye, Mengmeng Wang, Tianxin Huang, Guang Dai, Ivor W Tsang, Yong Liu.
Decentralized Riemannian conjugate gradient method on the Stiefel manifold.
International Conference on Learning Representations(ICLR),2024.
[pdf] - Luo Luo, Cheng Chen, Guangzeng Xie, Haishan Ye.
Revisiting Co-Occurring Directions: Sharper Analysis and Efficient Algorithm for Sparse Matrices.
Proceedings of the AAAI Conference on Artificial Intelligence(AAAI),2021.
[pdf] - Rui Pan, Haishan Ye*†, Tong Zhang.
Eigencurve: Optimal learning rate schedule for sgd on quadratic objectives with skewed hessian spectrums.
International Conference on Learning Representations(ICLR),2021.
[pdf] - Dachao Lin, Haishan Ye†, Zhihua Zhang.
Greedy and random quasi-newton methods with faster explicit superlinear convergence.
Advances in Neural Information Processing Systems(NeurIPS),2021.
[pdf] - Haishan Ye, Ziang Zhou, Luo Luo, Tong Zhang.
Decentralized accelerated proximal gradient descent.
Advances in Neural Information Processing Systems(NeurIPS),2020.
[pdf] - Luo Luo, Haishan Ye, Zhichao Huang, Tong Zhang.
Stochastic recursive gradient descent ascent for stochastic nonconvex-strongly-concave minimax problems.
Advances in Neural Information Processing Systems(NeurIPS),2020.
[pdf] - Chaoyang He, Haishan Ye†, Li Shen, Tong Zhang.
Milenas: Efficient neural architecture search via mixed-level reformulation.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(IEEE/CVF),2020.
[pdf] - Haishan Ye, Luo Luo, Zhihua Zhang.
Approximate Newton methods and their local convergence.
International Conference on Machine Learning, 2017.
[link] - Yujun Li, Kaichun Mo, Haishan Ye.
Accelerating random Kaczmarz algorithm based on clustering information.
Proceedings of the AAAI Conference on Artificial Intelligence(AAAI),2016.
[pdf]
Journal Publications
- Jun Shang, Haishan Ye*, Xiangyu Chang.
Accelerated double-sketching subspace Newton.
European Journal of Operational Research(EJOR),2024.
[link] - Haishan Ye, Luo Luo, Ziang Zhou, Tong Zhang.
Multi-consensus decentralized accelerated gradient descent.
Journal of Machine Learning Research(JMLR),2023.
[pdf] - Haishan Ye, Dachao Lin, Xiangyu Chang, Zhihua Zhang.
Towards explicit superlinear convergence rate for SR1.
Mathematical Programming,2023.
[link] - Haishan Ye, Shiyuan He, Xiangyu Chang.
DINE: Decentralized Inexact Newton With Exact Linear Convergence Rate.
IEEE Transactions on Signal Processing,2023.
[link] - Dachao Lin, Haishan Ye*, Zhihua Zhang.
Explicit convergence rates of greedy and random quasi-Newton methods.
Journal of Machine Learning Research(JMLR),2022.
[pdf] - Haishan Ye, Chaoyang He, Xiangyu Chang.
Accelerated distributed approximate Newton method.
IEEE Transactions on Neural Networks and Learning Systems,2022.
[link] - Haishan Ye, Luo Luo, Zhihua Zhang.
Approximate newton methods.
Journal of Machine Learning Research(JMLR),2021.
[pdf] - Haishan Ye*, Tong Zhang.
DeEPCA: Decentralized exact PCA with linear convergence rate.
Journal of Machine Learning Research(JMLR),2021.
[pdf] - Haishan Ye, Luo Luo, Zhihua Zhang.
Nesterov's acceleration for approximate Newton.
Journal of Machine Learning Research(JMLR),2020.
[pdf] - Haishan Ye, Luo Luo, Zhihua Zhang.
Accelerated proximal subsampled Newton method.
IEEE Transactions on Neural Networks and Learning Systems,2020.
[link] - Haishan Ye, Guangzeng Xie, Luo Luo, Zhihua Zhang.
Fast stochastic second-order method logarithmic in condition number.
Pattern Recognition,2019.
[link] - Haishan Ye, Yujun Li, Cheng Chen, Zhihua Zhang.
Fast Fisher discriminant analysis with randomized algorithms.
Pattern Recognition,2017.
[link]
Preprints
- Haishan Ye, Dachao Lin, Xiangyu Chang, Zhihua Zhang.
Anderson Acceleration Without Restart: A Novel Method with n-Step Super Quadratic Convergence Rate.
arXiv preprint arXiv:2403.16734.
[pdf] - Yanjun Zhao, Sizhe Dang,Haishan Ye*,Guang Dai, Yi Qian, Ivor W Tsang.
Second-Order Fine-Tuning without Pain for LLMs: A Hessian Informed Zeroth-Order Optimizer.
arXiv preprint arXiv:2402.15173.
[pdf] - Haishan Ye,Xiangyu Chang.
Optimal Decentralized Composite Optimization for Strongly Convex Functions.
arXiv preprint arXiv:2312.15845.
[pdf] - Hao Di, Yi Yang, Haishan Ye, Xiangyu Chang.
PPFL: A Personalized Federated Learning Framework for Heterogeneous Population.
arXiv preprint arXiv:2310.14337.
[pdf] - Haishan Ye.
Mirror natural evolution strategies.
arXiv preprint arXiv:2308.00469.
[pdf] - Lesi Chen, Haishan Ye, Luo Luo.
An Efficient Stochastic Algorithm for Decentralized Nonconvex-Strongly-Concave Minimax Optimization.
arXiv preprint arXiv:2212.02387.
[pdf] - Haishan Ye, Xiangyu Chang.
Snap-shot decentralized stochastic gradient tracking methods.
arXiv preprint arXiv:2212.05273.
[pdf] - Luo Luo, Haishan Ye.
An optimal stochastic algorithm for decentralized nonconvex finite-sum optimization.
arXiv preprint arXiv:2210.13931.
[pdf] - Luo Luo, Haishan Ye.
Decentralized stochastic variance reduced extragradient method.
arXiv preprint arXiv:2202.00509.
[pdf] - Haishan Ye, Dachao Lin, Zhihua Zhang.
Greedy and Random Broyden's Methods with Explicit Superlinear Convergence Rates in Nonlinear Equations.
arXiv preprint arXiv:2110.08572.
[pdf] - Dachao Lin, Haishan Ye, Zhihua Zhang.
Explicit superlinear convergence rates of Broyden's methods in nonlinear equations.
arXiv preprint arXiv:2109.01974.
[pdf] - Haishan Ye, Shusen Wang, Zhihua Zhang, Tong Zhang.
Fast Generalized Matrix Regression with Applications in Machine Learning.
arXiv preprint arXiv:1912.12008.
[pdf] - Haishan Ye, Zhichao Huang, Cong Fang, Chris Junchi Li, Tong Zhang.
Hessian-aware zeroth-order optimization for black-box adversarial attack.
arXiv preprint arXiv:1812.11377.
[pdf] - Haishan Ye,Wei Xiong, Tong Zhang.
PMGT-VR: A decentralized proximal-gradient algorithmic framework with variance reduction.
arXiv preprint arXiv:2012.15010.
[pdf]