Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

virtual machine istiod on debian shows permission denied #52716

Open
2 tasks done
sdake opened this issue Aug 15, 2024 · 4 comments
Open
2 tasks done

virtual machine istiod on debian shows permission denied #52716

sdake opened this issue Aug 15, 2024 · 4 comments
Labels
area/environments area/security feature/Virtual-machine issues related with VM support

Comments

@sdake
Copy link
Member

sdake commented Aug 15, 2024 •
edited by istio-policy-bot
Loading

Is this the right place to submit this?

  • This is not a security vulnerability or a crashing bug
  • This is not a question about how to use Istio

Bug Description

Following single network automatic workloadentry creation from this guide:
https://istio.io/latest/docs/setup/install/virtual-machine/#create-files-to-transfer-to-the-virtual-machine

$ sudo cat /var/log/istio/istio.log

2024-08-15T09:51:42.705907Z	info	FLAG: --concurrency="0"
2024-08-15T09:51:42.706157Z	info	FLAG: --domain=""
2024-08-15T09:51:42.706280Z	info	FLAG: --help="false"
2024-08-15T09:51:42.706289Z	info	FLAG: --log_as_json="false"
2024-08-15T09:51:42.706295Z	info	FLAG: --log_caller=""
2024-08-15T09:51:42.706301Z	info	FLAG: --log_output_level=""
2024-08-15T09:51:42.706307Z	info	FLAG: --log_rotate=""
2024-08-15T09:51:42.706313Z	info	FLAG: --log_rotate_max_age="30"
2024-08-15T09:51:42.706320Z	info	FLAG: --log_rotate_max_backups="1000"
2024-08-15T09:51:42.706326Z	info	FLAG: --log_rotate_max_size="104857600"
2024-08-15T09:51:42.706333Z	info	FLAG: --log_stacktrace_level="default:none"
2024-08-15T09:51:42.706355Z	info	FLAG: --log_target="[stdout]"
2024-08-15T09:51:42.706362Z	info	FLAG: --meshConfig="./etc/istio/config/mesh"
2024-08-15T09:51:42.706368Z	info	FLAG: --outlierLogPath=""
2024-08-15T09:51:42.706374Z	info	FLAG: --profiling="true"
2024-08-15T09:51:42.706380Z	info	FLAG: --proxyComponentLogLevel=""
2024-08-15T09:51:42.706386Z	info	FLAG: --proxyLogLevel="warning,misc:error"
2024-08-15T09:51:42.706393Z	info	FLAG: --serviceCluster="istio-proxy"
2024-08-15T09:51:42.706399Z	info	FLAG: --stsPort="0"
2024-08-15T09:51:42.706405Z	info	FLAG: --templateFile=""
2024-08-15T09:51:42.706411Z	info	FLAG: --tokenManagerPlugin=""
2024-08-15T09:51:42.706427Z	info	FLAG: --vklog="0"
2024-08-15T09:51:42.706434Z	info	Version 1.23.0-d0ca037e58f90f58b38b33c5769fec25b6bfa9f3-Clean
2024-08-15T09:51:42.706445Z	info	Set max file descriptors (ulimit -n) to: 1048576
2024-08-15T09:51:42.706854Z	info	Proxy role	ips=[192.168.34.160 172.17.0.1] type=sidecar id=wise-00.vllm domain=vllm.svc.cluster.local
2024-08-15T09:51:42.706894Z	info	Apply proxy config from env
serviceCluster: vllm.vllm
controlPlaneAuthPolicy: MUTUAL_TLS

2024-08-15T09:51:42.713794Z	warn	concurrency is set to 0, which will use a thread per CPU on the host. However, CPU limit is set lower. This is not recommended and may lead to performance issues. CPU count: 32, CPU Limit: 0.
2024-08-15T09:51:42.714219Z	info	Effective config: binaryPath: /usr/local/bin/envoy
configPath: ./etc/istio/proxy
controlPlaneAuthPolicy: MUTUAL_TLS
discoveryAddress: istiod.istio-system.svc:15012
drainDuration: 45s
proxyAdminPort: 15000
serviceCluster: vllm.vllm
statNameLength: 189
statusPort: 15020
terminationDrainDuration: 5s

2024-08-15T09:51:42.714238Z	info	JWT policy is third-party-jwt
2024-08-15T09:51:42.714246Z	info	using credential fetcher of JWT type in cluster.local trust domain
2024-08-15T09:51:42.917189Z	info	dns	Starting local udp DNS server on 127.0.0.1:15053
2024-08-15T09:51:42.917241Z	info	Opening status port 15020
2024-08-15T09:51:42.917465Z	info	dns	Starting local tcp DNS server on 127.0.0.1:15053
2024-08-15T09:51:42.917529Z	info	Starting default Istio SDS Server
2024-08-15T09:51:42.917544Z	info	CA Endpoint istiod.istio-system.svc:15012, provider Citadel
2024-08-15T09:51:42.917565Z	info	Using CA istiod.istio-system.svc:15012 cert with certs: /etc/certs/root-cert.pem
2024-08-15T09:51:42.918694Z	warn	Failed to create directory for var/run/secrets/workload-spiffe-uds/socket: mkdir var: permission denied
2024-08-15T09:51:42.918844Z	info	xdsproxy	Initializing with upstream address "istiod.istio-system.svc:15012" and cluster "cluster1"
2024-08-15T09:51:42.918903Z	warn	Failed to create directory for etc/istio/proxy/XDS: mkdir etc: permission denied
2024-08-15T09:51:42.918889Z	info	sds	Starting SDS grpc server
2024-08-15T09:51:42.920058Z	error	failed to start xds proxy: failed to listen on unix socket "etc/istio/proxy/XDS": listen unix etc/istio/proxy/XDS: bind: no such file or directory

There were two permission denied:

2024-08-15T09:51:42.918694Z	warn	Failed to create directory for var/run/secrets/workload-spiffe-uds/socket: mkdir var: permission denied
2024-08-15T09:51:42.918903Z	warn	Failed to create directory for etc/istio/proxy/XDS: mkdir etc: permission denied

It is unclear to me why the paths are not explicit (ie /var/run/..., instead they are var/run).

The net result is istiod in Debian does not start correctly.

Version

control plane version: 1.23.0
data plane version: 1.23.0 (1 proxies)
Client Version: v1.30.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.1

Additional Information

No response
@sdake
Copy link
Member Author

sdake commented Aug 15, 2024

This is actually a crashing bug. I wasn't sure how to file otherwise.

@sdake
Copy link
Member Author

sdake commented Aug 15, 2024 •
edited
Loading

Looking at istio-start.sh several paths are prepended with . eg:

# Load optional config variables
ISTIO_SIDECAR_CONFIG=${ISTIO_SIDECAR_CONFIG:-./var/lib/istio/envoy/sidecar.env}
if [[ -r ${ISTIO_SIDECAR_CONFIG} ]]; then
  # shellcheck disable=SC1090
  . "$ISTIO_SIDECAR_CONFIG"
fi

Why not specify the full path?

@sdake
Copy link
Member Author

sdake commented Aug 15, 2024

It is poor form for a daemon to create directories in /etc. Definately to be avoided.

@sdake
Copy link
Member Author

sdake commented Aug 16, 2024

@howardjohn answered that the full path isn't specified so that the tooling can be run locally without root permissions. I still believe there are better ways to achieve this. I am not being critical of the work done, because it is outstanding, but the istio-start.sh is difficult to read and understand.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/environments area/security feature/Virtual-machine issues related with VM support
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sdake @istio-policy-bot