Ok, I found it's (La?)TeX default, specifically: By default math fonts 
are italic, and what I want is upright font. So, currently I found it 
can be done by \usepackage{mathastext} that makes italic math font a 
upright font or by using "Neo Euler" font . At the moment I succesfully 
applied it to PDF export, both mathastext and neo euler. I think I'll 
stick to Neo Euler, it's more straightforward, although Libertinus looks 
better, unfortunately it has weird SEM symbol. Maybe if other symbols 
are more straightforward I'll switch back. However, I still don't know 
how to apply those font settings to latex previews in org-mode.

I found great article on beautifying org latex export: 
https://so.nwalsh.com/2020/01/05-latex Here is my improved version of 
those export options, they might be useful for someone: (I also attach 
init.el for rest of my latex config)

One note before: You need to install Liberation font on your systems if 
you want to use it, it is used in my config. Next thing - if you want 
upright Libertine instead of Neo Euler, change some lines to this 
(you'll notice which):
>   \\usepackage{mathastext}
>   \\usepackage{libertinus}
>   \\setmainfont{Liberation Serif}
>   \\setsansfont[Scale=MatchLowercase]{Liberation Sans}
>   \\setmonofont[Scale=MatchLowercase]{Liberation Mono}
>   \\setmathfont{Libertinus Math}

Config:
> (setq
>   org-format-latex-options (plist-put org-format-latex-options ':scale' 5)
>   org-latex-compiler "lualatex"
>   org-latex-listings t
>   org-latex-default-packages-alist
>     '(("" "graphicx" t)
>       ("" "grffile" t)
>       ("" "longtable" nil)
>       ("" "wrapfig" nil)
>       ("" "rotating" nil)
>       ("normalem" "ulem" t)
>       ("" "textcomp" t)
>       ("" "capt-of" nil)
>       ("" "hyperref" nil))
>
>   org-latex-classes
>   '(;; ("equation"
> ;; "\\usepackage{fontspec}
> ;;   \\setmainfont{Liberation Serif}
> ;;   \\setmathfont[Scale=MatchLowercase]{DejaVu Math TeX Gyre}
> ;;   \\setsansfont[Scale=MatchLowercase]{Raleway}
> ;;   \\setmonofont[Scale=MatchLowercase]{Operator Mono SSm}")
>   ("article"
> "\\RequirePackage{fix-cm}
>   \\PassOptionsToPackage{svgnames}{xcolor}
>   \\documentclass[14pt]{article}
>   \\usepackage{neo-euler}
>   \\setmainfont{Liberation Serif}
>   \\setsansfont[Scale=MatchLowercase]{Liberation Sans}
>   \\setmonofont[Scale=MatchLowercase]{Liberation Mono}
>   \\setmathfont{Neo Euler}
>   \\usepackage{sectsty}
>   \\allsectionsfont{\\sffamily}
>   \\usepackage{enumitem}
> \\setlist[description]{style=unboxed,font=\\sffamily\\bfseries}
>   \\usepackage{listings}
>   \\lstset{frame=single,aboveskip=1em,
>       framesep=.5em,backgroundcolor=\\color{AliceBlue},
>       rulecolor=\\color{LightSteelBlue},framerule=1pt}
>   \\usepackage{xcolor}
> \\newcommand\\basicdefault[1]{\\scriptsize\\color{Black}\\ttfamily#1}
>   \\lstset{basicstyle=\\basicdefault{\\spaceskip1em}}
>   \\lstset{literate=
>           keywordstyle=\\color{DarkGreen}\\bfseries,
>           identifierstyle=\\color{DarkRed},
>           commentstyle=\\color{Gray}\\upshape,
>           stringstyle=\\color{DarkBlue}\\upshape,
>           emphstyle=\\color{Chocolate}\\upshape,
>           showstringspaces=false,
>           columns=fullflexible,
>           keepspaces=true}
>   \\usepackage[a4paper,margin=0.5in]{geometry}
>   \\usepackage{parskip}
>   \\makeatletter
>   \\renewcommand{\\maketitle}{%
>     \\begingroup\\parindent0pt
>     \\sffamily
>     \\Huge{\\bfseries\\@title}\\par\\bigskip
>     \\LARGE{\\bfseries\\@author}\\par\\medskip
>     \\normalsize\\@date\\par\\bigskip
>     \\endgroup\\@afterindentfalse\\@afterheading}
>   \\makeatother
>   [DEFAULT-PACKAGES]
>   \\hypersetup{linkcolor=Blue,urlcolor=DarkBlue,
>     citecolor=DarkRed,colorlinks=true}
>   \\AtBeginDocument{\\renewcommand{\\UrlFont}{\\ttfamily}}
>   [PACKAGES]
>   [EXTRA]"
>   ("\\section{%s}" . "\\section*{%s}")
>   ("\\subsection{%s}" . "\\subsection*{%s}")
>   ("\\subsubsection{%s}" . "\\subsubsection*{%s}")
>   ("\\paragraph{%s}" . "\\paragraph*{%s}")
>   ("\\subparagraph{%s}" . "\\subparagraph*{%s}"))
>
>   ("report" "\\documentclass[11pt]{report}"
>   ("\\part{%s}" . "\\part*{%s}")
>   ("\\chapter{%s}" . "\\chapter*{%s}")
>   ("\\section{%s}" . "\\section*{%s}")
>   ("\\subsection{%s}" . "\\subsection*{%s}")
>   ("\\subsubsection{%s}" . "\\subsubsection*{%s}"))
>
>   ("book" "\\documentclass[11pt]{book}"
>   ("\\part{%s}" . "\\part*{%s}")
>   ("\\chapter{%s}" . "\\chapter*{%s}")
>   ("\\section{%s}" . "\\section*{%s}")
>   ("\\subsection{%s}" . "\\subsection*{%s}")
>   ("\\subsubsection{%s}" . "\\subsubsection*{%s}")))
>
> org-latex-pdf-process
>       (list (concat "latexmk -"
>                     org-latex-compiler
>                     " -recorder -synctex=1 %b"))
>       )

Credits of course go also to creators of this very nice template, I just 
improved it. In attachments there is example pdf produced with it.

btw: inline $...$ has broken fraction distances, use $$...$$ instead