分类 默认分类 下的文章

Java heap dump OQL samples - where

where 可以包含:

=, <=, >, <, [ NOT ] LIKE, [ NOT ] IN, IMPLEMENTS (relational operations)
AND OR != , =
字段可以 [. ] . .

实现 group by 功能

  1. 方法1: 如果是 group by 其中一个字段 可以这样: 菜单选择 -> Java Basics -> Group by Values -> 给出类名和需要group by 的字段:
    group1.png group2.png
  2. 方法2: 使用 OQL. 举例: 假如我有很多 brave.handler.MutableSpan, 这个类有个实例字段是 name, 我想根据 name 去分组. 我们需要这么做:

    // 第0步, 我们查看我们要分组的对象
    SELECT toString(s.name), * FROM brave.handler.MutableSpan s
    
    // 第一步, 我们看看有多少唯一的 name
    SELECT DISTINCT toString(s.name) FROM brave.handler.MutableSpan s
    
    // 第二步, 做分组, 把上一步distinct的结果和原始列表对比, 第二列返回的每一行是一个list
    SELECT dn.name AS name, (SELECT OBJECTS lst FROM brave.handler.MutableSpan s WHERE (toString(s.name) = dn.name)) AS lst
    FROM 
    OBJECTS (SELECT DISTINCT toString(s.name) AS name FROM brave.handler.MutableSpan s) dn
    
    // 第三步, count 每个 group, 第二步第二列是一个list, 所以可以使用 .@length 来取长度
    SELECT g.name as name, g.lst.@length as size FROM OBJECTS ( eval((
    
    SELECT dn.name AS name, (SELECT OBJECTS s FROM brave.handler.MutableSpan s WHERE (toString(s.name) = dn.name)) AS lst
    FROM 
    OBJECTS (SELECT DISTINCT toString(s.name) AS name FROM brave.handler.MutableSpan s) dn
    
    )) ) g
  3. 使用上面第一步的结果, 然后导出到csv(菜单栏 -> 最右边-> 导出CSV), 然后excel 操作

查询 URL 相关的:

- SELECT * FROM java.net.URL u where u.port = 443
- SELECT * FROM java.net.URL u where toString(u.host) = "api.google.com"
- SELECT * FROM java.net.URL u where u.@displayName like ".*api.google.com.*"
- SELECT * FROM "com.tianxiaohui.*" u where toString(u) like ".*Metrics.*"  //正则
- SELECT s.address.holder.hostName.toString(), s.timeout FROM java.net.SocksSocketImpl s WHERE (s.port = 443) //socket 的地址和 timeout 时间
- SELECT toString(u.string) FROM java.net.URI u WHERE (toString(u.schemeSpecificPart) LIKE ".+google.com.+")

查询其它相关的:

- SELECT x.capacity FROM java.nio.DirectByteBuffer x WHERE ((toString(x.att) = "null") and (toString(x.cleaner) != "null") and (x.capacity >= (1024 * 1024)))
- SELECT DISTINCT objects x.this$0 FROM java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask x 
- SELECT distinct objects x FROM java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask x WHERE (x.this$0.toString() LIKE ".+0x468369b50")
- SELECT * FROM INSTANCEOF java.lang.Object t WHERE (toHex(t.@objectAddress) >= "0xfbd4c000" AND toHex(t.@objectAddress) <= "0xfce94050") //一段地址空间中的所有对象
- select * from java.nio.DirectByteBuffer x where x.capacity > 65535 and x.cleaner != null
- SELECT o.toString() FROM OBJECTS ( SELECT OBJECTS outbounds(t) FROM org.apache.tomcat.util.threads.TaskThread t WHERE (t.toString() = "DefaultThreadPool-23") ) o WHERE (o.toString() LIKE ".+TracingInfoImpl.+")

如果 0x789342b78 对应的地址是一个类class(不是instance), 那么可以用下面这种查法:
select * from 0x789342b78

如果要查询某个 Class 的 static 字段里面的数据:

SELECT * FROM java.lang.Class x where x.toString() LIKE ".+com.tianxiaohui.platform.config.impl.ConfigProvider.*"

如果要查询一个抽象类的具体实现:

select * from INSTANCEOF java.net.AbstractPlainSocketImpl

Java JDBC 相关的

# 一个连接有多少 open 的 ResultSet
SELECT * FROM oracle.jdbc.driver.OracleResultSetImpl rs WHERE (rs.connection.toString() LIKE ".+0x7a3af6cf8")

查看是不是包含某个关键字的字符串String:

SELECT * FROM java.lang.String s where s.toString() like ".*agepsvc.*"

查看栈桢信息

SELECT u.Thread AS Thread, u.Frame.@text AS Frame 
  FROM OBJECTS ( 
    SELECT t AS Thread, ${snapshot}.getThreadStack(t.@objectId).@stackFrames AS Frame 
      FROM java.lang.Thread t  ) u

过滤栈上面的局部变量:

SELECT vr.Thread, vr.Name, vr.Frame, vr.Local FROM OBJECTS ( 
    SELECT v.Thread AS Thread, toString(v.Thread) AS Name, v.Frame AS Frame, ${snapshot}.getObject(v.Objs) AS Local FROM OBJECTS ( 
        SELECT u.Thread AS Thread, u.Frame.@text AS Frame, u.Frame.@localObjectsIds AS Objs FROM OBJECTS ( 
            SELECT t AS Thread, ${snapshot}.getThreadStack(t.@objectId).@stackFrames AS Frame FROM INSTANCEOF java.lang.Thread t WHERE (toString(t.name) = "DefaultThreadPool-32") 
        ) u  
    ) v WHERE (v.Objs != null) 
) vr WHERE (vr.Local.toString() LIKE ".*TracingInfoImpl.*")

内置函数:

  1. toHex( number ) //转换数字为16进制
  2. toString( object ) //转换对象为 String
  3. dominators( object ) //被这个对象直接控制的
  4. dominatorof( object ) //这个对象被那些对象直接控制
  5. outbounds( object ) //
  6. inbounds( object ) //
  7. classof( object ) // 当前对象的类

如何查看一个对象的 dominator, 然后就可以用 dominatorof() 函数:

SELECT dominatorof(x) FROM OBJECTS 15038294 x 
SELECT * FROM org.ebayopensource.ginger.core.logging.impl.CalLogTransactionImpl x WHERE (dominatorof(x).toString() = "DefaultThreadPool-20")

dominator.png

更多OQL的官方文档参考: https://wiki.eclipse.org/MemoryAnalyzer/OQL

MAT Java 内存分析工具

当前 MAT 可以分析 HPROF 二进制(produced by Sun, HP, SAP, etc… JVMs) 和 IBM system dumps (after preprocessing them), 以及 IBM portable heap dumps (PHD) .

  • find the biggest objects, as MAT provides reasonable accumulated size (retained size)
  • explore the object graph, both inbound and outbound references
  • compute paths from the garbage collector roots to interesting objects
  • find memory waste, like redundant String objects, empty collection objects, etc...

如何获得 heap dump

note: 从JDK 6 update 14 and above, HPROF 里面也包含所有线程的 callstatck.
参考: Heap Dump Analysis with Memory Analyzer, Part 1: Heap Dumps

如何分析 IBM J9 JVM dump?
https://help.eclipse.org/2020-03/index.jsp?topic=/org.eclipse.mat.ui.help/welcome.html

netstat 命令参数

This program is obsolete. Replacement for netstat is ss. Replacement for netstat -r is ip route. Replacement for netstat -i is ip -s link. Replacement for netstat -g is ip maddr.

netstat -t -l 查看监听的 tcp
netstat -t --wide
netstat -an |grep :8080 端口8080 上的连接 (有些外部的)

只针对 linux, Mac 和 win 有些不一样.

  • a all
  • r 显示路由表
  • s statistics
  • n 不做主机和端口转换, 数字形式 number
  • c continuous print
  • e extend 多显示 owner
  • p 显示 program
  • l listening
    --wide 不截取

状态:
ESTABLISHED
The socket has an established connection.
SYN_SENT
The socket is actively attempting to establish a connection.
SYN_RECV
A connection request has been received from the network.
FIN_WAIT1
The socket is closed, and the connection is shutting down.
FIN_WAIT2
Connection is closed, and the socket is waiting for a shutdown from the remote end.
TIME_WAIT
The socket is waiting after close to handle packets still in the network.
CLOSED
The socket is not being used.
CLOSE_WAIT
The remote end has shut down, waiting for the socket to close.
LAST_ACK
The remote end has shut down, and the socket is closed. Waiting for acknowledgement.
LISTEN
The socket is listening for incoming connections. Such sockets are not included in the output unless you specify the --listening (-l) or --all (-a) option.
CLOSING
Both sockets are shut down but we still don't have all our data sent.
UNKNOWN
The state of the socket is unknown.

https://en.wikipedia.org/wiki/Netstat
https://linux.die.net/man/8/netstat
https://www.computerhope.com/unix/unetstat.htm

docker internal

if you look in the Linux kernel, there is no such thing as a container

  • Containers share the host kernel
  • Containers use the kernel ability to group processes for resource control
  • Containers ensure isolation through namespaces
  • Containers feel like lightweight VMs (lower footprint, faster)

    history

  • Chroot circa 1982
  • FreeBSD Jails circa 2000
  • Solaris Zones circa 2004
  • Meiosys - MetaClusters with Checkpoint/Restore 2004-05
  • Linux OpenVZ circa 2005 (not in mainstream Linux)
  • AIX WPARs circa 2007
  • LXC circa 2008
  • Systemd-nspawn circa 2010-2013
  • Docker circa 2013
    -- built on LXC
    -- moved to libcontainer (March 2014)
    -- appC (CoreOS) announced (December 2014)
    -- Open Containers standard for convergence with Docker Announced (June 2015)
    -- moved to runC (OCF compliant) (July 2015)

    how it works

    Namespaces, cgroups, Images, Layers & copy-on-write

    Kernel Namespaces: isolation

  • Process trees (PID Namespace)
  • Mounts (MNT namespace) wc -l /proc/mounts
  • Network (Net namespace) ip addr
  • Users / UIDs (User Namespace)
  • Hostnames (UTS Namespace) hostname
  • Inter Process Communication (IPC Namespace) ipcs

    Control Group: accounting

    Kernel control groups (cgroups) allow you to do accounting on resources used by processes, a little bit of access control on device nodes and other things such as freezing groups of processes.

    IPTables (networking)

    solation on the networking level is achieved through the creation of virtual switches in the linux kernel. Linux Bridge is a kernel module, first introduced in 2.2 kernel (circa 2000). And it is administered using the brctl command on Linux.

Types of Containers

Given the above constructs, containers may be divided into 3 types as follows:

  1. System Containers share rootfs, PID, network, IPC and UTS with host system but live inside a cgroup.
  2. Application Containers live inside a cgroup and use namespaces (PID, network, IPC, chroot) for isolation from host system
  3. Pods use namespaces for isolation from host system but create sub groups which share PID, network, IPC and UTS except the rootfs.

docker providing

  • Image management
  • Resource Isolation
  • File System Isolation
  • Network Isolation
  • Change Management
  • Sharing
  • Process Management
  • Service Discovery (DNS since 1.10)

refer:

  1. https://docs.docker.com/engine/docker-overview/
  2. http://docker-saigon.github.io/post/Docker-Internals/
  3. https://www.youtube.com/watch?v=sK5i-N34im8