Skip to content

CLOUDSTACK-8829 : Consecutive cold migration fails. - #2056

Merged
karuturi merged 1 commit into
apache:masterfrom
priyankparihar:CLOUDSTACK-8829
May 8, 2017
Merged

CLOUDSTACK-8829 : Consecutive cold migration fails.#2056
karuturi merged 1 commit into
apache:masterfrom
priyankparihar:CLOUDSTACK-8829

Conversation

@priyankparihar

@priyankparihar priyankparihar commented Apr 20, 2017

Copy link
Copy Markdown
Contributor

Issue - Consecutive VM cold migration fails.
Root Cause Analysis - In case of VMware, if VM is being cold migrated between clusters belonging to two different VMware DCs, Cloudstack unregisters the VM from the source host and cleans up the associated VM files. The check if a VM is being cold migrated across DCs is made using the source host id. In case of consecutive cold migrations since the source host id of a VM is NULL and no VM exists, Cloudstack should skip the check
Proposed Solution - Attempt to unregister a VM in another DC, only if there is a host associated with a VM.

This closes #797

@priyankparihar

Copy link
Copy Markdown
Contributor Author

Hi @borisstoyanov Bhai,
I need one help. Could you please run relevent test cases on this PR ?

@borisstoyanov

Copy link
Copy Markdown
Contributor

@priyankparihar I can trigger smoketests for that.
@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-684

@borisstoyanov

Copy link
Copy Markdown
Contributor

@blueorangutan test centos7 vmware5.5u3

@blueorangutan

Copy link
Copy Markdown

@borisstoyanov unsupported parameters provided. Supported mgmt server os are: centos6, centos7, ubuntu. Supported hypervisors are: kvm-centos6, kvm-centos7, kvm-ubuntu, xenserver-65sp1, xenserver-62sp1, vmware-60u2, vmware-55u3, vmware-51u1, vmware-50u1

@borisstoyanov

Copy link
Copy Markdown
Contributor

@blueorangutan test centos7 vmware-55u3

@blueorangutan

Copy link
Copy Markdown

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + vmware-55u3) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-1044)
Environment: vmware-55u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43632 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2056-t1044-vmware-55u3.zip
Intermitten failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermitten failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermitten failure detected: /marvin/tests/smoke/test_network_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Test completed. 48 look ok, 3 have error(s)

Test Result Time (s) Test File
test_04_rvpc_privategw_static_routes Failure 893.79 test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup Failure 197.32 test_privategw_acl.py
test_02_vpc_privategw_static_routes Failure 146.87 test_privategw_acl.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers Error 242.31 test_vpc_redundant.py
test_06_verify_guest_lspci_again Error 0.00 test_deploy_virtio_scsi_vm.py
test_05_change_vm_ostype_restart Error 0.00 test_deploy_virtio_scsi_vm.py
test_04_verify_guest_lspci Error 0.00 test_deploy_virtio_scsi_vm.py
test_03_verify_libvirt_attach_disk Error 0.00 test_deploy_virtio_scsi_vm.py
test_02_verify_libvirt_after_restart Error 0.00 test_deploy_virtio_scsi_vm.py
test_01_verify_libvirt Error 0.00 test_deploy_virtio_scsi_vm.py
ContextSuite context=TestDeployVirtioSCSIVM>:teardown Error 0.00 test_deploy_virtio_scsi_vm.py
test_01_vpc_site2site_vpn Success 335.86 test_vpc_vpn.py
test_01_vpc_remote_access_vpn Success 156.81 test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpn Success 558.25 test_vpc_vpn.py
test_02_VPC_default_routes Success 319.34 test_vpc_router_nics.py
test_01_VPC_nics_after_destroy Success 691.35 test_vpc_router_nics.py
test_05_rvpc_multi_tiers Success 646.78 test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics Success 1539.26 test_vpc_redundant.py
test_02_redundant_VPC_default_routes Success 644.06 test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL Success 1335.05 test_vpc_redundant.py
test_09_delete_detached_volume Success 20.82 test_volumes.py
test_06_download_detached_volume Success 65.57 test_volumes.py
test_05_detach_volume Success 100.28 test_volumes.py
test_04_delete_attached_volume Success 15.22 test_volumes.py
test_03_download_attached_volume Success 10.28 test_volumes.py
test_02_attach_volume Success 58.75 test_volumes.py
test_01_create_volume Success 445.21 test_volumes.py
test_change_service_offering_for_vm_with_snapshots Success 459.08 test_vm_snapshots.py
test_03_delete_vm_snapshots Success 275.25 test_vm_snapshots.py
test_02_revert_vm_snapshots Success 199.11 test_vm_snapshots.py
test_01_create_vm_snapshots Success 163.69 test_vm_snapshots.py
test_deploy_vm_multiple Success 293.12 test_vm_life_cycle.py
test_deploy_vm Success 0.03 test_vm_life_cycle.py
test_advZoneVirtualRouter Success 0.03 test_vm_life_cycle.py
test_10_attachAndDetach_iso Success 26.82 test_vm_life_cycle.py
test_09_expunge_vm Success 125.23 test_vm_life_cycle.py
test_08_migrate_vm Success 101.45 test_vm_life_cycle.py
test_07_restore_vm Success 0.12 test_vm_life_cycle.py
test_06_destroy_vm Success 5.13 test_vm_life_cycle.py
test_03_reboot_vm Success 5.14 test_vm_life_cycle.py
test_02_start_vm Success 20.25 test_vm_life_cycle.py
test_01_stop_vm Success 5.12 test_vm_life_cycle.py
test_CreateTemplateWithDuplicateName Success 236.69 test_templates.py
test_08_list_system_templates Success 0.03 test_templates.py
test_07_list_public_templates Success 0.06 test_templates.py
test_05_template_permissions Success 0.07 test_templates.py
test_04_extract_template Success 20.65 test_templates.py
test_03_delete_template Success 5.13 test_templates.py
test_02_edit_template Success 90.15 test_templates.py
test_01_create_template Success 120.95 test_templates.py
test_10_destroy_cpvm Success 241.82 test_ssvm.py
test_09_destroy_ssvm Success 233.64 test_ssvm.py
test_08_reboot_cpvm Success 156.69 test_ssvm.py
test_07_reboot_ssvm Success 158.53 test_ssvm.py
test_06_stop_cpvm Success 176.95 test_ssvm.py
test_05_stop_ssvm Success 213.82 test_ssvm.py
test_04_cpvm_internals Success 1.12 test_ssvm.py
test_03_ssvm_internals Success 3.38 test_ssvm.py
test_02_list_cpvm_vm Success 0.13 test_ssvm.py
test_01_list_sec_storage_vm Success 0.14 test_ssvm.py
test_02_list_snapshots_with_removed_data_store Success 197.74 test_snapshots.py
test_01_snapshot_root_disk Success 26.33 test_snapshots.py
test_04_change_offering_small Success 98.24 test_service_offerings.py
test_03_delete_service_offering Success 0.04 test_service_offerings.py
test_02_edit_service_offering Success 0.08 test_service_offerings.py
test_01_create_service_offering Success 0.10 test_service_offerings.py
test_02_sys_template_ready Success 0.13 test_secondary_storage.py
test_01_sys_vm_start Success 0.19 test_secondary_storage.py
test_09_reboot_router Success 120.79 test_routers.py
test_08_start_router Success 110.76 test_routers.py
test_07_stop_router Success 25.25 test_routers.py
test_06_router_advanced Success 0.06 test_routers.py
test_05_router_basic Success 0.04 test_routers.py
test_04_restart_network_wo_cleanup Success 5.66 test_routers.py
test_03_restart_network_cleanup Success 130.98 test_routers.py
test_02_router_internal_adv Success 1.09 test_routers.py
test_01_router_internal_basic Success 0.57 test_routers.py
test_router_dns_guestipquery Success 77.40 test_router_dns.py
test_router_dns_externalipquery Success 0.08 test_router_dns.py
test_router_dhcphosts Success 141.15 test_router_dhcphosts.py
test_router_dhcp_opts Success 21.67 test_router_dhcphosts.py
test_01_updatevolumedetail Success 0.10 test_resource_detail.py
test_01_reset_vm_on_reboot Success 30.35 test_reset_vm_on_reboot.py
test_createRegion Success 0.04 test_regions.py
test_create_pvlan_network Success 5.23 test_pvlan.py
test_dedicatePublicIpRange Success 0.46 test_public_ip_range.py
test_01_vpc_privategw_acl Success 193.07 test_privategw_acl.py
test_03_migration_options_storage_tags Success 37.21 test_primary_storage.py
test_02_edit_primary_storage_tags Success 0.12 test_primary_storage.py
test_01_primary_storage_nfs Success 37.31 test_primary_storage.py
test_01_deploy_vms_storage_tags Success 46.62 test_primary_storage.py
test_createPortablePublicIPRange Success 15.25 test_portable_publicip.py
test_createPortablePublicIPAcquire Success 15.52 test_portable_publicip.py
test_isolate_network_password_server Success 59.18 test_password_server.py
test_UpdateStorageOverProvisioningFactor Success 0.13 test_over_provisioning.py
test_oobm_zchange_password Success 30.86 test_outofbandmanagement.py
test_oobm_multiple_mgmt_server_ownership Success 16.43 test_outofbandmanagement.py
test_oobm_issue_power_status Success 10.28 test_outofbandmanagement.py
test_oobm_issue_power_soft Success 15.42 test_outofbandmanagement.py
test_oobm_issue_power_reset Success 15.40 test_outofbandmanagement.py
test_oobm_issue_power_on Success 15.41 test_outofbandmanagement.py
test_oobm_issue_power_off Success 10.34 test_outofbandmanagement.py
test_oobm_issue_power_cycle Success 15.35 test_outofbandmanagement.py
test_oobm_enabledisable_across_clusterzones Success 82.71 test_outofbandmanagement.py
test_oobm_enable_feature_valid Success 5.21 test_outofbandmanagement.py
test_oobm_enable_feature_invalid Success 0.10 test_outofbandmanagement.py
test_oobm_disable_feature_valid Success 5.22 test_outofbandmanagement.py
test_oobm_disable_feature_invalid Success 0.10 test_outofbandmanagement.py
test_oobm_configure_invalid_driver Success 0.08 test_outofbandmanagement.py
test_oobm_configure_default_driver Success 0.08 test_outofbandmanagement.py
test_oobm_background_powerstate_sync Success 29.45 test_outofbandmanagement.py
test_extendPhysicalNetworkVlan Success 15.33 test_non_contigiousvlan.py
test_01_nic Success 520.59 test_nic.py
test_releaseIP Success 318.54 test_network.py
test_reboot_router Success 604.90 test_network.py
test_public_ip_user_account Success 10.33 test_network.py
test_public_ip_admin_account Success 40.36 test_network.py
test_network_rules_acquired_public_ip_3_Load_Balancer_Rule Success 77.19 test_network.py
test_network_rules_acquired_public_ip_2_nat_rule Success 61.80 test_network.py
test_network_rules_acquired_public_ip_1_static_nat_rule Success 124.71 test_network.py
test_delete_account Success 277.99 test_network.py
test_02_port_fwd_on_non_src_nat Success 55.81 test_network.py
test_01_port_fwd_on_src_nat Success 111.86 test_network.py
test_nested_virtualization_vmware Success 424.40 test_nested_virtualization.py
test_nic_secondaryip_add_remove Success 293.34 test_multipleips_per_nic.py
test_list_zones_metrics Success 0.26 test_metrics_api.py
test_list_volumes_metrics Success 5.45 test_metrics_api.py
test_list_vms_metrics Success 187.01 test_metrics_api.py
test_list_pstorage_metrics Success 0.35 test_metrics_api.py
test_list_infrastructure_metrics Success 0.44 test_metrics_api.py
test_list_hosts_metrics Success 0.48 test_metrics_api.py
test_list_clusters_metrics Success 0.37 test_metrics_api.py
login_test_saml_user Success 19.32 test_login.py
test_assign_and_removal_lb Success 148.12 test_loadbalance.py
test_02_create_lb_rule_non_nat Success 207.03 test_loadbalance.py
test_01_create_lb_rule_src_nat Success 207.83 test_loadbalance.py
test_03_list_snapshots Success 0.09 test_list_ids_parameter.py
test_02_list_templates Success 0.04 test_list_ids_parameter.py
test_01_list_volumes Success 0.03 test_list_ids_parameter.py
test_07_list_default_iso Success 0.07 test_iso.py
test_05_iso_permissions Success 0.06 test_iso.py
test_04_extract_Iso Success 5.18 test_iso.py
test_03_delete_iso Success 95.25 test_iso.py
test_02_edit_iso Success 0.08 test_iso.py
test_01_create_iso Success 21.00 test_iso.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces Success 490.08 test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfaces Success 359.02 test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 Success 968.72 test_internal_lb.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 Success 807.22 test_internal_lb.py
test_dedicateGuestVlanRange Success 10.27 test_guest_vlan_range.py
test_UpdateConfigParamWithScope Success 0.14 test_global_settings.py
test_rolepermission_lifecycle_update Success 6.19 test_dynamicroles.py
test_rolepermission_lifecycle_list Success 6.03 test_dynamicroles.py
test_rolepermission_lifecycle_delete Success 6.00 test_dynamicroles.py
test_rolepermission_lifecycle_create Success 5.91 test_dynamicroles.py
test_rolepermission_lifecycle_concurrent_updates Success 6.05 test_dynamicroles.py
test_role_lifecycle_update_role_inuse Success 5.92 test_dynamicroles.py
test_role_lifecycle_update Success 11.04 test_dynamicroles.py
test_role_lifecycle_list Success 5.92 test_dynamicroles.py
test_role_lifecycle_delete Success 10.97 test_dynamicroles.py
test_role_lifecycle_create Success 5.92 test_dynamicroles.py
test_role_inuse_deletion Success 5.87 test_dynamicroles.py
test_role_account_acls_multiple_mgmt_servers Success 8.29 test_dynamicroles.py
test_role_account_acls Success 8.60 test_dynamicroles.py
test_default_role_deletion Success 6.04 test_dynamicroles.py
test_04_create_fat_type_disk_offering Success 0.08 test_disk_offerings.py
test_03_delete_disk_offering Success 0.04 test_disk_offerings.py
test_02_edit_disk_offering Success 0.06 test_disk_offerings.py
test_02_create_sparse_type_disk_offering Success 0.10 test_disk_offerings.py
test_01_create_disk_offering Success 0.10 test_disk_offerings.py
test_deployvm_userdispersing Success 81.02 test_deploy_vms_with_varied_deploymentplanners.py
test_deployvm_userconcentrated Success 55.83 test_deploy_vms_with_varied_deploymentplanners.py
test_deployvm_firstfit Success 166.37 test_deploy_vms_with_varied_deploymentplanners.py
test_deployvm_userdata_post Success 30.53 test_deploy_vm_with_userdata.py
test_deployvm_userdata Success 126.31 test_deploy_vm_with_userdata.py
test_02_deploy_vm_root_resize Success 5.87 test_deploy_vm_root_resize.py
test_01_deploy_vm_root_resize Success 5.85 test_deploy_vm_root_resize.py
test_00_deploy_vm_root_resize Success 6.00 test_deploy_vm_root_resize.py
test_deploy_vm_from_iso Success 187.85 test_deploy_vm_iso.py
test_3d_gpu_support Success 445.43 test_deploy_vgpu_enabled_vm.py
test_DeployVmAntiAffinityGroup Success 176.92 test_affinity_groups.py
test_08_resize_volume Skipped 5.12 test_volumes.py
test_07_resize_fail Skipped 10.28 test_volumes.py
test_06_copy_template Skipped 0.00 test_templates.py
test_static_role_account_acls Skipped 0.03 test_staticroles.py
test_11_ss_nfs_version_on_ssvm Skipped 0.02 test_ssvm.py
test_01_scale_vm Skipped 63.32 test_scale_vm.py
test_01_primary_storage_iscsi Skipped 0.04 test_primary_storage.py
test_06_copy_iso Skipped 0.00 test_iso.py
test_deploy_vgpu_enabled_vm Skipped 1.34 test_deploy_vgpu_enabled_vm.py

@borisstoyanov borisstoyanov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the virtio test failures will be fixed in a separate FR

@sateesh-chodapuneedi sateesh-chodapuneedi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for code changes.

@DaanHoogland DaanHoogland left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes look good

} catch (final Exception e) {
throw new CloudRuntimeException("Failed to unregister VM: " + vm.getInstanceName() + " from source host: " + srcHost.getId() +
_agentMgr.send(srcHostId, uvc);
} catch (final AgentUnavailableException | OperationTimedoutException e) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yay

@karuturi
karuturi merged commit 08c138d into apache:master May 8, 2017
@karuturi karuturi added this to the 4.10.0.0 milestone May 8, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants