src/test: add libcephfs tests for async(nonblocking) calls #54435

dparmar18 · 2023-11-09T15:35:32Z

add async i/o test cases

Fixes: https://tracker.ceph.com/issues/63104
Signed-off-by: Dhairya Parmar dparmar@redhat.com

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins retest this please
jenkins test classic perf
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard cephadm
jenkins test api
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox
jenkins test windows
jenkins test rook e2e

src/test/client/nonblocking.cc

github-actions · 2024-01-30T04:45:31Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

qa/suites/fs/libcephfs/tasks/client_full.yaml

src/test/client/nonblocking_full.cc

vshankar

LGTM

dparmar18 · 2024-07-09T11:38:54Z

hey venky, the testcases in 55f5cb6 aren't making use of the helpers that in f5f0f3c because the helpers were introduced after i wrote those test cases and this thing went unnoticed. I'll have to change this and I think this should be the final change (I promise :P).

EDIT: please see #54435 (comment)

this is temp, will be removed once ceph#54435 is merged Signed-off-by: Dhairya Parmar <dparmar@redhat.com> (cherry picked from commit f5f0f3c)

dparmar18 · 2024-07-09T14:44:26Z

hey venky, the testcases in 55f5cb6 aren't making use of the helpers that in f5f0f3c because the helpers were introduced after i wrote those test cases and this thing went unnoticed. I'll have to change this and I think this should be the final change (I promise :P).

this is just a cleanup and doesn't impact the code in any other way, since you've already reviewed it and we have a cleanup tracker waiting already for nonblocking test cases, i think i can do this cleanup in that tracker itself. So I think this is good to go for testing.

This is a dependency for a call in the test case but exists in ceph#54435. This commit will be removed once that PR is merged. Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

vshankar · 2024-07-18T18:35:17Z

This PR is under test in https://tracker.ceph.com/issues/67020.

vshankar · 2024-08-05T06:19:17Z

qa/suites/fs/libcephfs/tasks/client_full.yaml

+- workunit:
+    clients:
+      client.0:
+        - client/test_full.sh


@dparmar18 What mode is test_full.sh created with? I ask since this fails with:

File "/home/teuthworker/src/git.ceph.com_ceph-c_5f7f38f6943da1b8f6e6fe9aed0202920c629e98/qa/tasks/workunit.py", line 388, in _run_tests raise RuntimeError('Spec did not match any workunits: {spec!r}'.format(spec=spec)) RuntimeError: Spec did not match any workunits: 'client/test_full.sh'

See: /a/vshankar-2024-08-01_09:58:56-fs-wip-vshankar-testing-20240801.064407-debug-testing-default-smithi/7829964

(Oh lord) Fixed it now

vshankar

PTAL at the failure.

dparmar18 · 2024-08-05T10:43:26Z

PTAL at the failure.

fixed

vshankar · 2024-08-05T10:47:13Z

src/test/client/nonblocking.cc

@@ -611,3 +612,499 @@ TEST_F(TestClient, LlreadvLlwritevInvalidFileHandle) {
  ASSERT_EQ(bytes_read, -CEPHFS_EBADF);
  ASSERT_EQ(bl.length(), 0);
 }
+
+TEST_F(TestClient, LlreadvLlwritevOverlimit) {


/a/vshankar-2024-08-01_09:58:56-fs-wip-vshankar-testing-20240801.064407-debug-testing-default-smithi/7829908

I see the following crash

INFO:tasks.workunit.client.0.smithi142.stderr: -1> 2024-08-02T07:09:20.600+0000 7f987f000640 20 client.4689 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-08-02T07:08:01.737249+0000 mtime=2024-08-02T07:09:20.594231+0000 ctime=2024-08-02T07:09:20.594231+0000 change_attr=18 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f986000c330) n = 1 2024-08-02T07:09:20.717 INFO:tasks.workunit.client.0.smithi142.stderr: 0> 2024-08-02T07:09:20.601+0000 7f988880c480 -1 *** Caught signal (Aborted) ** 2024-08-02T07:09:20.717 INFO:tasks.workunit.client.0.smithi142.stderr: in thread 7f988880c480 thread_name:ceph_test_clien 2024-08-02T07:09:20.717 INFO:tasks.workunit.client.0.smithi142.stderr: 2024-08-02T07:09:20.717 INFO:tasks.workunit.client.0.smithi142.stderr: ceph version 19.0.0-5453-g5f7f38f6 (5f7f38f6943da1b8f6e6fe9aed0202920c629e98) squid (dev) 2024-08-02T07:09:20.717 INFO:tasks.workunit.client.0.smithi142.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f988863e6f0] 2024-08-02T07:09:20.717 INFO:tasks.workunit.client.0.smithi142.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f988868b94c] 2024-08-02T07:09:20.717 INFO:tasks.workunit.client.0.smithi142.stderr: 3: raise() 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 4: abort() 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 5: /lib64/libc.so.6(+0x29130) [0x7f9888629130] 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 6: /lib64/libc.so.6(+0x11d77b) [0x7f988871d77b] 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 7: /lib64/libc.so.6(+0x11c0e6) [0x7f988871c0e6] 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 8: __strcat_chk() 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 9: (TestClient_LlreadvLlwritevOverlimit_Test::TestBody()+0x2c4) [0x55e475d8bd5c] 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 10: ceph_test_client(+0x1fee96) [0x55e475ec2e96] 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 11: ceph_test_client(+0x204b8c) [0x55e475ec8b8c] 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 12: (testing::Test::Run()+0xb8) [0x55e475ea60a2] 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 13: (testing::TestInfo::Run()+0x108) [0x55e475ea61ac] 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 14: (testing::TestSuite::Run()+0xb6) [0x55e475ea6264] 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 15: (testing::internal::UnitTestImpl::RunAllTests()+0x2dd) [0x55e475eb8be5] 2024-08-02T07:09:20.718 INFO:tasks.workunit.client.0.smithi142.stderr: 16: ceph_test_client(+0x1feef6) [0x55e475ec2ef6] 2024-08-02T07:09:20.719 INFO:tasks.workunit.client.0.smithi142.stderr: 17: ceph_test_client(+0x205026) [0x55e475ec9026] 2024-08-02T07:09:20.719 INFO:tasks.workunit.client.0.smithi142.stderr: 18: (testing::UnitTest::Run()+0x95) [0x55e475eb48a3] 2024-08-02T07:09:20.719 INFO:tasks.workunit.client.0.smithi142.stderr: 19: main() 2024-08-02T07:09:20.719 INFO:tasks.workunit.client.0.smithi142.stderr: 20: /lib64/libc.so.6(+0x29590) [0x7f9888629590] 2024-08-02T07:09:20.719 INFO:tasks.workunit.client.0.smithi142.stderr: 21: __libc_start_main() 2024-08-02T07:09:20.719 INFO:tasks.workunit.client.0.smithi142.stderr: 22: _start()

it's complaining about string concatenation, should be this line strcat(out_str, num); in TEST_F(TestClient, LlreadvLlwritevOverlimit).

As per https://refspecs.linuxbase.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/libc---strcat-chk-1.html: __strcat_chk() shall check for buffer overflow before computing a result. If an overflow is anticipated, the function shall abort and the program calling it shall exit.

Looks like there was buffer overflow

@vshankar simplied the test case. This shouldn't bother now.

github-actions · 2024-08-22T09:42:01Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

This is a dependency for a call in the test case but exists in ceph#54435. This commit will be removed once that PR is merged. Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

Fixes: https://tracker.ceph.com/issues/63104 Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

This requires a new suite, cannot be run with other async i/o test cases therefore apart from adding the test case, add a binary, a shell script to run it and a YAML file to pick it up in teuthology. Fixes: https://tracker.ceph.com/issues/63104 Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

dparmar18 · 2024-10-09T10:35:00Z

@vshankar PTAL, fixed the conflicts

this is temp, will be removed once ceph#54435 is merged Signed-off-by: Dhairya Parmar <dparmar@redhat.com> (cherry picked from commit f5f0f3c)

github-actions bot added the tests label Nov 9, 2023

dparmar18 mentioned this pull request Nov 9, 2023

src/test: add libcephfs tests for async(nonblocking) calls #54243

Closed

14 tasks

dparmar18 force-pushed the libcephfs-nonblocking-io-testcases branch 4 times, most recently from c0ef42b to 2e7f65f Compare November 10, 2023 21:01

dparmar18 force-pushed the libcephfs-nonblocking-io-testcases branch 15 times, most recently from cb22b10 to cc60032 Compare November 27, 2023 10:36

vshankar reviewed Dec 1, 2023

View reviewed changes

src/test/client/nonblocking.cc Outdated Show resolved Hide resolved

dparmar18 force-pushed the libcephfs-nonblocking-io-testcases branch 3 times, most recently from b1aac0e to be7ab35 Compare December 18, 2023 13:02

dparmar18 force-pushed the libcephfs-nonblocking-io-testcases branch 2 times, most recently from 3e2898a to d429557 Compare January 11, 2024 13:26

github-actions bot added the needs-rebase label Jan 30, 2024

dparmar18 force-pushed the libcephfs-nonblocking-io-testcases branch from d429557 to ba7868c Compare February 5, 2024 11:58

vshankar reviewed Jul 8, 2024

View reviewed changes

qa/suites/fs/libcephfs/tasks/client_full.yaml Show resolved Hide resolved

vshankar reviewed Jul 9, 2024

View reviewed changes

src/test/client/nonblocking_full.cc Show resolved Hide resolved

vshankar added the needs-qa label Jul 9, 2024

vshankar reviewed Jul 9, 2024

View reviewed changes

vshankar added the wip-vshankar-testing3 label Jul 16, 2024

vshankar reviewed Aug 5, 2024

View reviewed changes

vshankar requested changes Aug 5, 2024

View reviewed changes

vshankar removed the wip-vshankar-testing3 label Aug 5, 2024

dparmar18 force-pushed the libcephfs-nonblocking-io-testcases branch from 542ae49 to 1b844d4 Compare August 5, 2024 10:42

vshankar requested changes Aug 5, 2024

View reviewed changes

dparmar18 force-pushed the libcephfs-nonblocking-io-testcases branch from 1b844d4 to 7c42357 Compare August 6, 2024 18:22

github-actions bot added the needs-rebase label Aug 22, 2024

dparmar18 mentioned this pull request Sep 27, 2024

client: Fix libcephfs aio metadata corruption. #59987

Merged

14 tasks

dparmar18 added 3 commits October 9, 2024 16:01

src/test: add some helper functions to avoid cumbersomeness

a90ed04

Fixes: https://tracker.ceph.com/issues/63104 Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

src/test: async i/o test cases

70ee583

Fixes: https://tracker.ceph.com/issues/63104 Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

dparmar18 force-pushed the libcephfs-nonblocking-io-testcases branch from 7c42357 to fbff10a Compare October 9, 2024 10:34

github-actions bot removed the needs-rebase label Oct 9, 2024

dparmar18 requested a review from vshankar October 9, 2024 10:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

src/test: add libcephfs tests for async(nonblocking) calls #54435

src/test: add libcephfs tests for async(nonblocking) calls #54435

dparmar18 commented Nov 9, 2023 •

edited

Loading

github-actions bot commented Jan 30, 2024

vshankar left a comment

dparmar18 commented Jul 9, 2024 •

edited

Loading

dparmar18 commented Jul 9, 2024 •

edited

Loading

vshankar commented Jul 18, 2024

vshankar Aug 5, 2024

dparmar18 Aug 5, 2024

vshankar left a comment

dparmar18 commented Aug 5, 2024

vshankar Aug 5, 2024

dparmar18 Aug 5, 2024

dparmar18 Aug 6, 2024

github-actions bot commented Aug 22, 2024

dparmar18 commented Oct 9, 2024

src/test: add libcephfs tests for async(nonblocking) calls #54435

Are you sure you want to change the base?

src/test: add libcephfs tests for async(nonblocking) calls #54435

Conversation

dparmar18 commented Nov 9, 2023 • edited Loading

Contribution Guidelines

Checklist

github-actions bot commented Jan 30, 2024

vshankar left a comment

Choose a reason for hiding this comment

dparmar18 commented Jul 9, 2024 • edited Loading

dparmar18 commented Jul 9, 2024 • edited Loading

vshankar commented Jul 18, 2024

vshankar Aug 5, 2024

Choose a reason for hiding this comment

dparmar18 Aug 5, 2024

Choose a reason for hiding this comment

vshankar left a comment

Choose a reason for hiding this comment

dparmar18 commented Aug 5, 2024

vshankar Aug 5, 2024

Choose a reason for hiding this comment

dparmar18 Aug 5, 2024

Choose a reason for hiding this comment

dparmar18 Aug 6, 2024

Choose a reason for hiding this comment

github-actions bot commented Aug 22, 2024

dparmar18 commented Oct 9, 2024

dparmar18 commented Nov 9, 2023 •

edited

Loading

dparmar18 commented Jul 9, 2024 •

edited

Loading

dparmar18 commented Jul 9, 2024 •

edited

Loading